2020-09-12T13:46:07.761 INFO:root:teuthology version: 1.0.0-55bd2dc 2020-09-12T13:46:07.763 DEBUG:teuthology.run:Teuthology command: teuthology --block --archive /home/teuthworker/archive/kchai-2020-09-12_13:41:38-rados-wip-kefu-testing-2020-09-12-2016-distro-basic-smithi/5428298 --verbose --owner scheduled_kchai@teuthology --name kchai-2020-09-12_13:41:38-rados-wip-kefu-testing-2020-09-12-2016-distro-basic-smithi --lock --description rados/standalone/{supported-random-distro$/{centos_8} workloads/osd} -- /tmp/teuthology-worker.1zcgs3ym.tmp 2020-09-12T13:46:07.781 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-09-12T13:46:07.829 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/kchai-2020-09-12_13:41:38-rados-wip-kefu-testing-2020-09-12-2016-distro-basic-smithi/5428298 branch: wip-kefu-testing-2020-09-12-2016 description: rados/standalone/{supported-random-distro$/{centos_8} workloads/osd} email: tchaikov@gmail.com first_in_suite: false job_id: '5428298' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: kchai-2020-09-12_13:41:38-rados-wip-kefu-testing-2020-09-12-2016-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 3 size: 10 os_type: centos os_version: '8.1' overrides: admin_socket: branch: wip-kefu-testing-2020-09-12-2016 ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: f346cc73d15a568d3bec1c99a58953de76f57b85 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: f346cc73d15a568d3bec1c99a58953de76f57b85 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-kefu-testing-2020-09-12-2016 sha1: f346cc73d15a568d3bec1c99a58953de76f57b85 owner: scheduled_kchai@teuthology priority: 90 repo: git://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 sha1: f346cc73d15a568d3bec1c99a58953de76f57b85 sleep_before_teardown: 0 suite: rados suite_branch: wip-kefu-testing-2020-09-12-2016 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-09-12-2016/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: f346cc73d15a568d3bec1c99a58953de76f57b85 tasks: - install: null - workunit: basedir: qa/standalone clients: all: - osd teuthology_branch: master timestamp: 2020-09-12_13:41:38 tube: smithi user: kchai verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.2784 2020-09-12T13:46:07.829 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-09-12-2016/qa; will attempt to use it 2020-09-12T13:46:07.830 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-09-12-2016/qa/tasks 2020-09-12T13:46:07.830 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-09-12T13:46:07.835 INFO:teuthology.task.internal:Checking packages... 2020-09-12T13:46:07.856 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'basic' and ceph hash 'f346cc73d15a568d3bec1c99a58953de76f57b85' 2020-09-12T13:46:07.857 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-09-12T13:46:07.857 INFO:teuthology.packaging:ref: None 2020-09-12T13:46:07.857 INFO:teuthology.packaging:tag: None 2020-09-12T13:46:07.858 INFO:teuthology.packaging:branch: wip-kefu-testing-2020-09-12-2016 2020-09-12T13:46:07.858 INFO:teuthology.packaging:sha1: f346cc73d15a568d3bec1c99a58953de76f57b85 2020-09-12T13:46:07.858 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-kefu-testing-2020-09-12-2016 2020-09-12T13:46:08.020 INFO:teuthology.task.internal:Found packages for ceph version 16.0.0-5485.gf346cc73d15 2020-09-12T13:46:08.021 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-09-12T13:46:08.033 INFO:teuthology.task.internal:no buildpackages task found 2020-09-12T13:46:08.033 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-09-12T13:46:08.055 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-09-12T13:46:08.056 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-09-12T13:46:08.106 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_kchai@teuthology', 'count': 1, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/kchai-2020-09-12_13:41:38-rados-wip-kefu-testing-2020-09-12-2016-distro-basic-smithi/5428298'} 2020-09-12T13:46:08.128 DEBUG:teuthology.lock.ops:locked ubuntu@smithi027.front.sepia.ceph.com 2020-09-12T13:46:08.162 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa3109bc2f0>, signals=[15]) 2020-09-12T13:46:08.355 INFO:teuthology.provision.fog.smithi027:Scheduling deploy of centos 8.1 2020-09-12T13:46:08.729 INFO:teuthology.orchestra.console:Power off smithi027 2020-09-12T13:46:08.731 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-09-12T13:46:08.759 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-12T13:46:12.786 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-12T13:46:16.898 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-12T13:46:17.110 INFO:teuthology.orchestra.console:Power off for smithi027 completed 2020-09-12T13:46:17.211 INFO:teuthology.orchestra.console:Power on smithi027 2020-09-12T13:46:17.212 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-09-12T13:46:17.249 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-12T13:46:21.310 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-12T13:46:25.424 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-12T13:46:25.636 INFO:teuthology.orchestra.console:Power on for smithi027 completed 2020-09-12T13:46:25.737 INFO:teuthology.provision.fog.smithi027:Waiting for deploy to finish 2020-09-12T13:48:41.926 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-12T13:48:50.994 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-12T13:49:00.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-12T13:49:09.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-12T13:49:18.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-12T13:49:27.218 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-12T13:49:58.961 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-09-12T13:49:58.963 INFO:teuthology.orchestra.run.smithi027:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-09-12T13:50:13.853 INFO:teuthology.provision.fog.smithi027:Node is ready 2020-09-12T13:50:13.853 INFO:teuthology.orchestra.run.smithi027:> hostname 2020-09-12T13:50:13.873 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027.front.sepia.ceph.com 2020-09-12T13:50:13.874 INFO:teuthology.orchestra.run.smithi027:> grep smithi027.front.sepia.ceph.com /etc/hosts 2020-09-12T13:50:13.930 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27 smithi027.front.sepia.ceph.com smithi027 2020-09-12T13:50:13.931 INFO:teuthology.orchestra.run.smithi027:> sudo hostname smithi027 2020-09-12T13:50:14.062 INFO:teuthology.orchestra.run.smithi027:> sudo sed -i -e 's/smithi027.front.sepia.ceph.com/smithi027/g' /etc/hosts 2020-09-12T13:50:14.097 INFO:teuthology.orchestra.run.smithi027:> sudo sed -i -e 's/172.21.15.27/172.21.15.27/g' /etc/hosts 2020-09-12T13:50:14.171 INFO:teuthology.orchestra.run.smithi027:> sudo sed -i -e 's/smithi027.front.sepia.ceph.com/smithi027/g' /etc/hostname 2020-09-12T13:50:14.244 INFO:teuthology.orchestra.run.smithi027:> sudo hostname smithi027 2020-09-12T13:50:14.315 INFO:teuthology.provision.fog.smithi027:Deploy complete! 2020-09-12T13:50:14.316 DEBUG:teuthology.parallel:result is None 2020-09-12T13:50:14.432 INFO:teuthology.lock.ops:Checking smithi027.front.sepia.ceph.com 2020-09-12T13:50:14.433 INFO:teuthology.lock.ops:New key found. Updating... 2020-09-12T13:50:14.448 INFO:teuthology.lock.ops:Updating [smithi027.front.sepia.ceph.com]: set os type and version on server 2020-09-12T13:50:14.449 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-09-12T13:50:14.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-12T13:50:14.518 INFO:teuthology.orchestra.run.smithi027:> true 2020-09-12T13:50:14.688 INFO:teuthology.orchestra.run.smithi027:> uname -m 2020-09-12T13:50:14.742 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2020-09-12T13:50:14.742 INFO:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2020-09-12T13:50:14.796 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Linux" 2020-09-12T13:50:14.797 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="8 (Core)" 2020-09-12T13:50:14.797 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2020-09-12T13:50:14.797 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2020-09-12T13:50:14.797 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="8" 2020-09-12T13:50:14.797 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el8" 2020-09-12T13:50:14.798 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-09-12T13:50:14.798 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2020-09-12T13:50:14.798 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-09-12T13:50:14.798 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://www.centos.org/" 2020-09-12T13:50:14.799 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-09-12T13:50:14.799 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:50:14.799 INFO:teuthology.orchestra.run.smithi027.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-09-12T13:50:14.799 INFO:teuthology.orchestra.run.smithi027.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-09-12T13:50:14.800 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-09-12T13:50:14.800 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-09-12T13:50:14.800 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:50:14.802 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2020-09-12T13:50:14.818 INFO:teuthology.task.internal.lock_machines:1 smithi machines locked this try, 1/1 locked so far 2020-09-12T13:50:14.835 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi027.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNXatoydS58kR1q0jK242G40rgddyqFhTPSpeZ72FVsI3bfVwgiF6SRc8jXhK4qgJITU/+E6toLTQMI2spuDVUjqVPqKs35BiqGqJXw3I8ypVsTMQ+nfiUmsYa/YiwuMQMYb86lZDMHei57OZApkZgm2u57eHQ81Yv6yUmjJHROePVJc+UBJreZbnUNPRDmKpptqKkaTWuszitnppBNov1oTWvWETS1otOWD1HPrycnQZe99pcZiHs94l4nxEF3xf4qL9f4vA3KVhex/HPqsvhs48xuTElI71m101Ad2myRjwNYKaZ83ImjqqX/yCceIGHHPbExoVOeuRV3nX8lssAn/gbcdVHcX64A5KqPHqi6FJA6pT/xOyGkYnnnM/kAaQrxrUfdYHgADQ4hJjt8l5r2NGVnAtIJhjw3RH6EOYvPgC855uqM6bPX5ypLv7zOSWWzWRL9TkHxvvCxTUU0n06U7azVsoWuznN6VEdS5WBv1ouKPCjswD1O55cF2c/iqc= 2020-09-12T13:50:14.836 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-09-12T13:50:14.879 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-09-12T13:50:14.888 INFO:teuthology.task.internal:Saving configuration 2020-09-12T13:50:14.897 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-09-12T13:50:14.905 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-09-12T13:50:14.921 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-09-12 13:46:08.115787', 'description': '/home/teuthworker/archive/kchai-2020-09-12_13:41:38-rados-wip-kefu-testing-2020-09-12-2016-distro-basic-smithi/5428298', 'locked_by': 'scheduled_kchai@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPgb96rvZusATgcbyXLp0f9zDqDlUxL14YJR7nfm32KLI5+JyyaRwHaiVlv6EOig3Q/FYcI74G6nos87wTwUJx0=', 'os_version': '8', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi027.front.sepia.ceph.com'} 2020-09-12T13:50:14.921 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-09-12T13:50:14.929 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2020-09-12T13:50:14.930 INFO:teuthology.run_tasks:Running task console_log... 2020-09-12T13:50:14.967 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa31346f268>, signals=[15]) 2020-09-12T13:50:14.967 INFO:teuthology.run_tasks:Running task internal.connect... 2020-09-12T13:50:14.978 INFO:teuthology.task.internal:Opening connections... 2020-09-12T13:50:14.978 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2020-09-12T13:50:14.980 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-12T13:50:15.052 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-09-12T13:50:15.063 INFO:teuthology.orchestra.run.smithi027:> uname -m 2020-09-12T13:50:15.221 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2020-09-12T13:50:15.222 INFO:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2020-09-12T13:50:15.275 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Linux" 2020-09-12T13:50:15.276 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="8 (Core)" 2020-09-12T13:50:15.276 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2020-09-12T13:50:15.276 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2020-09-12T13:50:15.276 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="8" 2020-09-12T13:50:15.276 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el8" 2020-09-12T13:50:15.277 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-09-12T13:50:15.277 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2020-09-12T13:50:15.277 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-09-12T13:50:15.277 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://www.centos.org/" 2020-09-12T13:50:15.277 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-09-12T13:50:15.277 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:50:15.277 INFO:teuthology.orchestra.run.smithi027.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-09-12T13:50:15.278 INFO:teuthology.orchestra.run.smithi027.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-09-12T13:50:15.278 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-09-12T13:50:15.278 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-09-12T13:50:15.278 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:50:15.278 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2020-09-12T13:50:15.295 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-09-12T13:50:15.311 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-09-12T13:50:15.338 INFO:teuthology.task.internal:Checking for old test directory... 2020-09-12T13:50:15.338 INFO:teuthology.orchestra.run.smithi027:> test '!' -e /home/ubuntu/cephtest 2020-09-12T13:50:15.352 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-09-12T13:50:15.362 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-09-12T13:50:15.362 INFO:teuthology.orchestra.run.smithi027:> test -z $(ls -A /var/lib/ceph) 2020-09-12T13:50:15.418 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-09-12T13:50:15.453 INFO:teuthology.run_tasks:Running task kernel... 2020-09-12T13:50:15.474 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}} 2020-09-12T13:50:15.475 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2020-09-12T13:50:15.475 INFO:teuthology.orchestra.run.smithi027:> uname -r 2020-09-12T13:50:15.489 INFO:teuthology.orchestra.run.smithi027.stdout:4.18.0-193.14.2.el8_2.x86_64 2020-09-12T13:50:15.490 INFO:teuthology.task.kernel:Running kernel on smithi027: 4.18.0-193.14.2.el8_2.x86_64 2020-09-12T13:50:15.490 INFO:teuthology.orchestra.run.smithi027:> sudo yum install -y kernel 2020-09-12T13:50:18.104 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-8 - AppStream 33 MB/s | 7.0 MB 00:00 2020-09-12T13:50:19.446 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-8 - Base 24 MB/s | 2.2 MB 00:00 2020-09-12T13:50:19.912 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-8 - Extras 77 kB/s | 5.5 kB 00:00 2020-09-12T13:50:20.304 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 23 MB/s | 8.0 MB 00:00 2020-09-12T13:50:22.081 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 3.7 MB/s | 21 kB 00:00 2020-09-12T13:50:23.084 INFO:teuthology.orchestra.run.smithi027.stdout:Package kernel-4.18.0-147.el8.x86_64 is already installed. 2020-09-12T13:50:23.084 INFO:teuthology.orchestra.run.smithi027.stdout:Package kernel-4.18.0-193.14.2.el8_2.x86_64 is already installed. 2020-09-12T13:50:23.263 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:50:23.264 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2020-09-12T13:50:23.264 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:50:23.384 INFO:teuthology.orchestra.run.smithi027:> echo no | sudo yum reinstall kernel || true 2020-09-12T13:50:23.819 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:01 ago on Sat 12 Sep 2020 01:50:22 PM UTC. 2020-09-12T13:50:24.084 INFO:teuthology.orchestra.run.smithi027.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-09-12T13:50:24.084 INFO:teuthology.orchestra.run.smithi027.stdout:Installed package kernel-4.18.0-193.14.2.el8_2.x86_64 (from BaseOS) not available. 2020-09-12T13:50:24.144 INFO:teuthology.orchestra.run.smithi027.stderr:Error: No packages marked for reinstall. 2020-09-12T13:50:24.191 INFO:teuthology.orchestra.run.smithi027:> sudo yum reinstall -y kernel || true 2020-09-12T13:50:24.622 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:02 ago on Sat 12 Sep 2020 01:50:22 PM UTC. 2020-09-12T13:50:24.892 INFO:teuthology.orchestra.run.smithi027.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-09-12T13:50:24.892 INFO:teuthology.orchestra.run.smithi027.stdout:Installed package kernel-4.18.0-193.14.2.el8_2.x86_64 (from BaseOS) not available. 2020-09-12T13:50:24.924 INFO:teuthology.orchestra.run.smithi027.stderr:Error: No packages marked for reinstall. 2020-09-12T13:50:24.963 INFO:teuthology.orchestra.run.smithi027:> rpm -q kernel --last | head -n 1 2020-09-12T13:50:25.034 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-4.18.0-193.14.2.el8_2.x86_64 Tue 01 Sep 2020 04:40:34 PM UTC 2020-09-12T13:50:25.036 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-193.14.2.el8_2.x86_64 2020-09-12T13:50:25.036 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-09-12T13:50:25.036 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2020-09-12T13:50:25.037 INFO:teuthology.orchestra.run.smithi027:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-09-12T13:50:25.075 INFO:teuthology.orchestra.run.smithi027.stdout:ttyS1 2020-09-12T13:50:25.088 INFO:teuthology.run_tasks:Running task internal.base... 2020-09-12T13:50:25.096 INFO:teuthology.task.internal:Creating test directory... 2020-09-12T13:50:25.097 INFO:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-09-12T13:50:25.142 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2020-09-12T13:50:25.184 INFO:teuthology.run_tasks:Running task internal.archive... 2020-09-12T13:50:25.207 INFO:teuthology.task.internal:Creating archive directory... 2020-09-12T13:50:25.208 INFO:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-09-12T13:50:25.232 INFO:teuthology.run_tasks:Running task internal.coredump... 2020-09-12T13:50:25.241 INFO:teuthology.task.internal:Enabling coredump saving... 2020-09-12T13:50:25.241 INFO:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-09-12T13:50:25.335 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T13:50:25.352 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T13:50:25.353 INFO:teuthology.run_tasks:Running task internal.sudo... 2020-09-12T13:50:25.361 INFO:teuthology.task.internal:Configuring sudo... 2020-09-12T13:50:25.362 INFO:teuthology.orchestra.run.smithi027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-09-12T13:50:25.394 INFO:teuthology.run_tasks:Running task internal.syslog... 2020-09-12T13:50:25.405 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2020-09-12T13:50:25.406 INFO:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-09-12T13:50:25.449 INFO:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-09-12T13:50:25.523 INFO:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-09-12T13:50:25.587 INFO:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-09-12T13:50:25.660 INFO:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-09-12T13:50:25.715 INFO:teuthology.orchestra.run.smithi027:> set -ex 2020-09-12T13:50:25.716 INFO:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-09-12T13:50:25.767 INFO:teuthology.orchestra.run.smithi027.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-09-12T13:50:25.783 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records in 2020-09-12T13:50:25.784 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records out 2020-09-12T13:50:25.784 INFO:teuthology.orchestra.run.smithi027.stderr:152 bytes copied, 3.0167e-05 s, 5.0 MB/s 2020-09-12T13:50:25.785 INFO:teuthology.orchestra.run.smithi027:> sudo service rsyslog restart 2020-09-12T13:50:25.905 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-09-12T13:50:26.224 INFO:teuthology.run_tasks:Running task internal.timer... 2020-09-12T13:50:26.245 INFO:teuthology.task.internal:Starting timer... 2020-09-12T13:50:26.246 INFO:teuthology.run_tasks:Running task pcp... 2020-09-12T13:50:26.279 INFO:teuthology.run_tasks:Running task selinux... 2020-09-12T13:50:26.320 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2020-09-12T13:50:26.321 INFO:teuthology.orchestra.run.smithi027:> sudo service auditd rotate 2020-09-12T13:50:26.394 INFO:teuthology.orchestra.run.smithi027.stdout:Rotating logs: [ OK ] 2020-09-12T13:50:26.396 DEBUG:teuthology.task.selinux:Getting current SELinux state 2020-09-12T13:50:26.396 INFO:teuthology.orchestra.run.smithi027:> /usr/sbin/getenforce 2020-09-12T13:50:26.417 INFO:teuthology.orchestra.run.smithi027.stdout:Permissive 2020-09-12T13:50:26.417 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi027.front.sepia.ceph.com': 'permissive'} 2020-09-12T13:50:26.418 INFO:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|scontext=system_u:system_r:logrotate_t:s0\)' 2020-09-12T13:50:26.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-09-12T13:50:26.490 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2020-09-12T13:50:26.490 INFO:teuthology.orchestra.run.smithi027:> sudo /usr/sbin/setenforce permissive 2020-09-12T13:50:26.560 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2020-09-12T13:50:26.569 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2020-09-12T13:50:26.570 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2020-09-12T13:50:26.582 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2020-09-12T13:50:26.583 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi027.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2020-09-12T13:50:26.900 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2020-09-12T13:50:27.737 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2020-09-12T13:50:29.684 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2020-09-12T13:50:29.702 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:29 +0000 (0:00:00.064) 0:00:00.064 **** 2020-09-12T13:50:32.361 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi027.front.sepia.ceph.com] 2020-09-12T13:50:32.426 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:32 +0000 (0:00:02.723) 0:00:02.788 **** 2020-09-12T13:50:32.858 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2020-09-12T13:50:32.860 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-09-12T13:50:32.891 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:32 +0000 (0:00:00.465) 0:00:03.253 **** 2020-09-12T13:50:33.394 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2020-09-12T13:50:33.396 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-09-12T13:50:33.425 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:33 +0000 (0:00:00.534) 0:00:03.787 **** 2020-09-12T13:50:33.451 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2020-09-12T13:50:33.453 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:33.483 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:33 +0000 (0:00:00.057) 0:00:03.845 **** 2020-09-12T13:50:33.916 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** ok: [smithi027.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-09-12T13:50:33.949 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:33 +0000 (0:00:00.465) 0:00:04.311 **** 2020-09-12T13:50:34.954 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2020-09-12T13:50:34.956 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1599918633.9638941-38346037599279/source", "state": "file", "uid": 0} 2020-09-12T13:50:34.992 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:34 +0000 (0:00:01.042) 0:00:05.354 **** 2020-09-12T13:50:35.476 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2020-09-12T13:50:35.477 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-09-12T13:50:35.479 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-09-12T13:50:35.652 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** 2020-09-12T13:50:35.653 INFO:teuthology.task.ansible.out:skipping: no hosts matched 2020-09-12T13:50:35.658 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2020-09-12T13:50:35.786 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:35 +0000 (0:00:00.794) 0:00:06.148 **** 2020-09-12T13:50:36.445 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi027.front.sepia.ceph.com] 2020-09-12T13:50:36.513 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2020-09-12T13:50:36.515 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:36 +0000 (0:00:00.727) 0:00:06.875 **** 2020-09-12T13:50:36.972 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2020-09-12T13:50:36.974 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.142915", "end": "2020-09-12 13:50:36.943680", "rc": 0, "start": "2020-09-12 13:50:36.800765", "stderr": "", "stderr_lines": [], "stdout": "d7c0190f3c79f0107823a9b3caef24ede18680e0", "stdout_lines": ["d7c0190f3c79f0107823a9b3caef24ede18680e0"]} 2020-09-12T13:50:37.007 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:37 +0000 (0:00:00.493) 0:00:07.369 **** 2020-09-12T13:50:37.230 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2020-09-12T13:50:37.231 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.013556", "end": "2020-09-12 13:50:37.209795", "failed_when_result": false, "rc": 0, "start": "2020-09-12 13:50:37.196239", "stderr": "", "stderr_lines": [], "stdout": "d7c0190f3c79f0107823a9b3caef24ede18680e0", "stdout_lines": ["d7c0190f3c79f0107823a9b3caef24ede18680e0"]} 2020-09-12T13:50:37.263 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:37 +0000 (0:00:00.255) 0:00:07.625 **** 2020-09-12T13:50:37.300 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-09-12T13:50:37.302 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-09-12T13:50:37.331 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:37 +0000 (0:00:00.068) 0:00:07.693 **** 2020-09-12T13:50:37.357 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-09-12T13:50:37.359 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.389 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:37 +0000 (0:00:00.056) 0:00:07.750 **** 2020-09-12T13:50:37.414 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-09-12T13:50:37.416 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.445 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:37 +0000 (0:00:00.057) 0:00:07.807 **** 2020-09-12T13:50:37.472 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-09-12T13:50:37.473 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.507 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:37 +0000 (0:00:00.062) 0:00:07.870 **** 2020-09-12T13:50:37.534 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.565 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:37 +0000 (0:00:00.057) 0:00:07.927 **** 2020-09-12T13:50:37.591 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2020-09-12T13:50:37.593 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.624 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:37 +0000 (0:00:00.059) 0:00:07.986 **** 2020-09-12T13:50:37.667 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-09-12T13:50:37.669 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.679 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.680 INFO:teuthology.task.ansible.out: skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.691 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.693 INFO:teuthology.task.ansible.out: skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.693 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.704 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.705 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.715 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.717 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.727 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.729 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.739 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.740 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.750 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.752 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.762 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.763 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.773 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.774 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.774 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.785 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.786 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.796 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.798 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.808 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.809 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.819 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.830 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.831 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.831 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.842 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.844 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.853 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.855 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.866 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.877 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.878 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.888 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.889 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.899 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.900 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.910 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.912 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.922 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.924 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.933 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.935 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.945 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.946 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.956 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.958 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.968 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.970 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.979 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.981 INFO:teuthology.task.ansible.out: skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.991 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:37.993 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.003 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.004 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.014 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.015 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.026 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.027 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.037 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.038 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.048 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.050 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.059 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.061 INFO:teuthology.task.ansible.out: skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.071 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.073 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.073 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.073 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.084 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.087 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.095 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.097 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.107 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.109 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.119 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.119 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.131 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.142 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.143 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.153 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.154 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.164 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.166 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.176 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.177 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.187 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.188 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.199 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.201 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.210 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.212 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.221 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.222 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.232 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.234 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.244 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.246 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.256 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.257 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.267 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.269 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.279 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.280 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.291 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.338 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:38 +0000 (0:00:00.713) 0:00:08.700 **** 2020-09-12T13:50:38.364 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-09-12T13:50:38.394 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:38 +0000 (0:00:00.056) 0:00:08.756 **** 2020-09-12T13:50:38.420 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2020-09-12T13:50:38.422 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.456 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:38 +0000 (0:00:00.061) 0:00:08.818 **** 2020-09-12T13:50:38.483 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.514 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:38 +0000 (0:00:00.057) 0:00:08.876 **** 2020-09-12T13:50:38.540 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-09-12T13:50:38.542 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.571 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:38 +0000 (0:00:00.057) 0:00:08.933 **** 2020-09-12T13:50:38.597 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-09-12T13:50:38.599 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.628 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:38 +0000 (0:00:00.056) 0:00:08.990 **** 2020-09-12T13:50:38.655 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2020-09-12T13:50:38.656 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.688 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:38 +0000 (0:00:00.059) 0:00:09.050 **** 2020-09-12T13:50:38.725 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-09-12T13:50:38.727 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.737 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.739 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.750 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.751 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.762 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.763 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.773 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.775 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.785 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.786 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.796 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.797 INFO:teuthology.task.ansible.out: skipping: [smithi027.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.808 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.810 INFO:teuthology.task.ansible.out: skipping: [smithi027.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.821 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.832 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.833 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.843 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.844 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.855 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.856 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.866 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.868 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.877 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.879 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.888 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.890 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.900 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.901 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.911 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.913 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.922 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.925 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:38.957 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:38 +0000 (0:00:00.269) 0:00:09.319 **** 2020-09-12T13:50:39.566 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-09-12T13:50:39.567 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "e4960b14bdbab8632f61f7e37ba0ea7d7809d19f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "state": "file", "uid": 0} 2020-09-12T13:50:39.599 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:39 +0000 (0:00:00.642) 0:00:09.961 **** 2020-09-12T13:50:39.626 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-09-12T13:50:39.628 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => { "msg": "Host smithi027.front.sepia.ceph.com is running CentOS 8.1 (Core)" } 2020-09-12T13:50:39.658 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:39 +0000 (0:00:00.058) 0:00:10.020 **** 2020-09-12T13:50:39.695 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-09-12T13:50:39.697 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml) => {"ansible_facts": {"nrpe_selinux_packages": ["python3-libsemanage", "python3-policycoreutils"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"} 2020-09-12T13:50:39.727 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:39 +0000 (0:00:00.068) 0:00:10.089 **** 2020-09-12T13:50:39.753 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-09-12T13:50:39.754 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:39.784 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:39 +0000 (0:00:00.057) 0:00:10.146 **** 2020-09-12T13:50:40.368 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-09-12T13:50:40.370 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.372899", "end": "2020-09-12 13:50:40.343278", "rc": 0, "start": "2020-09-12 13:50:39.970379", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-09-12T13:50:40.401 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:40 +0000 (0:00:00.617) 0:00:10.763 **** 2020-09-12T13:50:40.428 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-09-12T13:50:40.430 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:40.461 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:40 +0000 (0:00:00.060) 0:00:10.823 **** 2020-09-12T13:50:40.488 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-09-12T13:50:40.490 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:40.520 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:40 +0000 (0:00:00.058) 0:00:10.882 **** 2020-09-12T13:50:40.556 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2020-09-12T13:50:40.558 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:40.588 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:40 +0000 (0:00:00.067) 0:00:10.950 **** 2020-09-12T13:50:40.624 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-09-12T13:50:40.626 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:40.656 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:40 +0000 (0:00:00.068) 0:00:11.018 **** 2020-09-12T13:50:40.682 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-09-12T13:50:40.684 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:40.714 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:40 +0000 (0:00:00.057) 0:00:11.076 **** 2020-09-12T13:50:40.741 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:40.774 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:40 +0000 (0:00:00.060) 0:00:11.136 **** 2020-09-12T13:50:40.812 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-09-12T13:50:40.814 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi027.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-09-12T13:50:40.845 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:40 +0000 (0:00:00.070) 0:00:11.207 **** 2020-09-12T13:50:40.882 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-09-12T13:50:40.883 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:40.917 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:40 +0000 (0:00:00.071) 0:00:11.279 **** 2020-09-12T13:50:40.943 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2020-09-12T13:50:40.944 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:40.975 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:40 +0000 (0:00:00.058) 0:00:11.337 **** 2020-09-12T13:50:41.001 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2020-09-12T13:50:41.003 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-09-12T13:50:41.034 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:41 +0000 (0:00:00.058) 0:00:11.396 **** 2020-09-12T13:50:41.061 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-09-12T13:50:41.063 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:41.094 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:41 +0000 (0:00:00.059) 0:00:11.456 **** 2020-09-12T13:50:41.122 INFO:teuthology.task.ansible.out: 2020-09-12T13:50:41.124 INFO:teuthology.task.ansible.out:TASK [common : Register with subscription-manager.] **************************** skipping: [smithi027.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-09-12T13:50:41.154 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:41 +0000 (0:00:00.059) 0:00:11.516 **** 2020-09-12T13:50:41.180 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-09-12T13:50:41.182 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:41.213 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:41 +0000 (0:00:00.059) 0:00:11.575 **** 2020-09-12T13:50:41.240 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-09-12T13:50:41.242 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:41.272 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:41 +0000 (0:00:00.058) 0:00:11.634 **** 2020-09-12T13:50:41.298 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-09-12T13:50:41.300 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:41.330 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:41 +0000 (0:00:00.058) 0:00:11.692 **** 2020-09-12T13:50:41.356 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-09-12T13:50:41.358 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:41.389 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:41 +0000 (0:00:00.057) 0:00:11.750 **** 2020-09-12T13:50:41.414 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-09-12T13:50:41.416 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-09-12T13:50:41.445 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:41 +0000 (0:00:00.057) 0:00:11.807 **** 2020-09-12T13:50:41.473 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-09-12T13:50:41.475 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:41.504 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:41 +0000 (0:00:00.059) 0:00:11.866 **** 2020-09-12T13:50:41.531 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2020-09-12T13:50:41.532 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:41.562 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:41 +0000 (0:00:00.057) 0:00:11.924 **** 2020-09-12T13:50:41.588 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2020-09-12T13:50:41.590 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:41.620 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:41 +0000 (0:00:00.057) 0:00:11.982 **** 2020-09-12T13:50:41.648 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-09-12T13:50:41.650 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:41.680 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:41 +0000 (0:00:00.060) 0:00:12.042 **** 2020-09-12T13:50:41.717 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2020-09-12T13:50:41.719 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:41.753 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:41 +0000 (0:00:00.072) 0:00:12.115 **** 2020-09-12T13:50:41.780 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-09-12T13:50:41.781 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:41.811 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:41 +0000 (0:00:00.057) 0:00:12.173 **** 2020-09-12T13:50:41.838 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-09-12T13:50:41.840 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-09-12T13:50:41.869 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:41 +0000 (0:00:00.057) 0:00:12.231 **** 2020-09-12T13:50:41.895 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2020-09-12T13:50:41.898 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:41.928 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:41 +0000 (0:00:00.059) 0:00:12.290 **** 2020-09-12T13:50:41.954 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-09-12T13:50:41.955 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:41.987 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:41 +0000 (0:00:00.059) 0:00:12.349 **** 2020-09-12T13:50:42.222 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2020-09-12T13:50:42.224 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-09-12T13:50:42.255 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:42 +0000 (0:00:00.267) 0:00:12.617 **** 2020-09-12T13:50:42.758 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-09-12T13:50:42.760 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2020-09-12T13:50:43.225 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2020-09-12T13:50:43.258 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:43 +0000 (0:00:01.003) 0:00:13.620 **** 2020-09-12T13:50:43.752 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-09-12T13:50:43.753 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "3c9ba7bacd1e30bed014cefd1fb4eab0f0f8fa10", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 484, "state": "file", "uid": 0} 2020-09-12T13:50:44.220 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "d542c3e2c30a0d3eb7d454775342c302c0ded15b", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 540, "state": "file", "uid": 0} 2020-09-12T13:50:44.254 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:44 +0000 (0:00:00.996) 0:00:14.616 **** 2020-09-12T13:50:44.281 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:44.312 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:44 +0000 (0:00:00.057) 0:00:14.674 **** 2020-09-12T13:50:44.338 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-09-12T13:50:44.340 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:44.371 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:44 +0000 (0:00:00.058) 0:00:14.732 **** 2020-09-12T13:50:44.403 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-09-12T13:50:44.405 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:44.435 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:44 +0000 (0:00:00.064) 0:00:14.797 **** 2020-09-12T13:50:44.461 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-09-12T13:50:44.463 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:44.493 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:44 +0000 (0:00:00.057) 0:00:14.855 **** 2020-09-12T13:50:44.520 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-09-12T13:50:44.522 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:44.551 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:44 +0000 (0:00:00.058) 0:00:14.913 **** 2020-09-12T13:50:44.578 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:44.609 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:44 +0000 (0:00:00.057) 0:00:14.971 **** 2020-09-12T13:50:44.637 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:44.671 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:44 +0000 (0:00:00.062) 0:00:15.033 **** 2020-09-12T13:50:44.698 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-09-12T13:50:44.699 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:44.730 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:44 +0000 (0:00:00.058) 0:00:15.092 **** 2020-09-12T13:50:44.757 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:44.789 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:44 +0000 (0:00:00.059) 0:00:15.151 **** 2020-09-12T13:50:45.520 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-09-12T13:50:45.522 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.521707", "end": "2020-09-12 13:50:45.500344", "rc": 0, "start": "2020-09-12 13:50:44.978637", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-12T13:50:45.553 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:45 +0000 (0:00:00.764) 0:00:15.915 **** 2020-09-12T13:50:46.775 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-09-12T13:50:46.776 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: krb5-workstation"]} 2020-09-12T13:50:46.808 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:46 +0000 (0:00:01.254) 0:00:17.170 **** 2020-09-12T13:50:46.834 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-09-12T13:50:46.836 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:46.866 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:46 +0000 (0:00:00.058) 0:00:17.228 **** 2020-09-12T13:50:46.892 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-09-12T13:50:46.894 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:46.924 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:46 +0000 (0:00:00.057) 0:00:17.286 **** 2020-09-12T13:50:46.950 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:46.981 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:46 +0000 (0:00:00.057) 0:00:17.343 **** 2020-09-12T13:50:47.007 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-09-12T13:50:47.009 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:47.038 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:47 +0000 (0:00:00.057) 0:00:17.400 **** 2020-09-12T13:50:47.512 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-09-12T13:50:47.513 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2020-09-12T13:50:47.543 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:47 +0000 (0:00:00.504) 0:00:17.905 **** 2020-09-12T13:50:48.489 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-09-12T13:50:48.491 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: smartmontools"]} 2020-09-12T13:50:48.521 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:48 +0000 (0:00:00.977) 0:00:18.883 **** 2020-09-12T13:50:49.039 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-09-12T13:50:49.041 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2020-09-12T13:50:49.517 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2020-09-12T13:50:49.976 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2020-09-12T13:50:50.008 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:50 +0000 (0:00:01.487) 0:00:20.371 **** 2020-09-12T13:50:50.214 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-09-12T13:50:50.216 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-09-12T13:50:50.245 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:50 +0000 (0:00:00.236) 0:00:20.607 **** 2020-09-12T13:50:50.732 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-09-12T13:50:50.734 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2020-09-12T13:50:51.191 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2020-09-12T13:50:51.631 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2020-09-12T13:50:51.665 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:51 +0000 (0:00:01.419) 0:00:22.027 **** 2020-09-12T13:50:51.693 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-09-12T13:50:51.695 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:51.724 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:51 +0000 (0:00:00.059) 0:00:22.086 **** 2020-09-12T13:50:51.761 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-09-12T13:50:51.762 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2020-09-12T13:50:51.794 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:51 +0000 (0:00:00.069) 0:00:22.156 **** 2020-09-12T13:50:51.999 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-09-12T13:50:52.000 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-09-12T13:50:52.035 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:52 +0000 (0:00:00.241) 0:00:22.397 **** 2020-09-12T13:50:52.074 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** skipping: [smithi027.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:52.095 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:52.106 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:52.108 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:52.156 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:52 +0000 (0:00:00.120) 0:00:22.518 **** 2020-09-12T13:50:53.137 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-09-12T13:50:53.139 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: nagios-common", "Installed: nrpe", "Installed: nagios-plugins", "Installed: nagios-plugins-load"]} 2020-09-12T13:50:53.169 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:53 +0000 (0:00:01.012) 0:00:23.531 **** 2020-09-12T13:50:53.195 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-09-12T13:50:53.197 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:53.226 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:53 +0000 (0:00:00.057) 0:00:23.588 **** 2020-09-12T13:50:53.703 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-09-12T13:50:53.705 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2020-09-12T13:50:53.734 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:53 +0000 (0:00:00.507) 0:00:24.096 **** 2020-09-12T13:50:54.220 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-09-12T13:50:54.222 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2020-09-12T13:50:54.252 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:54 +0000 (0:00:00.517) 0:00:24.614 **** 2020-09-12T13:50:54.289 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ skipping: [smithi027.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:54.290 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2020-09-12T13:50:54.327 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:54 +0000 (0:00:00.074) 0:00:24.688 **** 2020-09-12T13:50:54.553 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-09-12T13:50:54.555 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-09-12T13:50:54.587 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:54 +0000 (0:00:00.260) 0:00:24.949 **** 2020-09-12T13:50:54.842 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** fatal: [smithi027.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2020-09-12T13:50:54.873 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:54 +0000 (0:00:00.286) 0:00:25.235 **** 2020-09-12T13:50:54.900 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-09-12T13:50:54.902 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:50:54.931 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:54 +0000 (0:00:00.058) 0:00:25.293 **** 2020-09-12T13:50:55.420 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "aa9150d4582f919d7a8504c01c8d86f05e3eeb5d", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1184, "state": "file", "uid": 0} 2020-09-12T13:50:55.451 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:55 +0000 (0:00:00.519) 0:00:25.813 **** 2020-09-12T13:50:56.189 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-09-12T13:50:56.190 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-09-12 13:49:54 UTC", "ActiveEnterTimestampMonotonic": "16033874", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "local-fs.target systemd-tmpfiles-setup.service systemd-journald.socket system.slice -.mount tmp.mount sysinit.target time-sync.target nss-lookup.target basic.target network.target var-run.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-09-12 13:49:54 UTC", "AssertTimestampMonotonic": "16032853", "Before": "plymouth-quit.service xdm.service shutdown.target getty@tty1.service multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-09-12 13:49:54 UTC", "ConditionTimestampMonotonic": "16032853", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "D 2020-09-12T13:50:56.192 INFO:teuthology.task.ansible.out:efaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2431", "ExecMainStartTimestamp": "Sat 2020-09-12 13:49:54 UTC", "ExecMainStartTimestampMonotonic": "16033839", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f ; ignore_errors=no ; start_time=[Sat 2020-09-12 13:49:54 UTC] ; stop_time=[n/a] ; pid=2431 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "GID": "986", "Group": "nrpe", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-09-12 13:49:54 UTC", "InactiveExitTimestampMonotonic": "16033874", "InvocationID": "717cd0283fb1465e95ed83af5de03f56", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "Limi 2020-09-12T13:50:56.193 INFO:teuthology.task.ansible.out:tDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127450", "LimitNPROCSoft": "127450", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127450", "LimitSIGPENDINGSoft": "127450", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2431", "MemoryAccounting": "yes", "MemoryCurrent": "1171456", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount system.slice sysinit.target", "RequiresMountsFor": "/var/tmp /run/nrpe", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectory": "n 2020-09-12T13:50:56.195 INFO:teuthology.task.ansible.out:rpe", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-09-12 13:49:54 UTC", "StateChangeTimestampMonotonic": "16033874", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "990", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-09-12 13:49:54 UTC", "WatchdogTimestampMonotonic": "16033873", "WatchdogUSec": "0"}} 2020-09-12T13:50:56.225 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:56 +0000 (0:00:00.773) 0:00:26.587 **** 2020-09-12T13:50:56.429 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-09-12T13:50:56.430 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001796", "end": "2020-09-12 13:50:56.406542", "rc": 0, "start": "2020-09-12 13:50:56.404746", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2020-09-12T13:50:56.462 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:56 +0000 (0:00:00.237) 0:00:26.824 **** 2020-09-12T13:50:57.461 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-09-12T13:50:57.463 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: python3-libsemanage", "Installed: python3-policycoreutils"]} 2020-09-12T13:50:57.497 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:57 +0000 (0:00:01.034) 0:00:27.858 **** 2020-09-12T13:50:58.468 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-09-12T13:50:58.470 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: selinux-policy-targeted"]} 2020-09-12T13:50:58.500 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:58 +0000 (0:00:01.003) 0:00:28.862 **** 2020-09-12T13:50:59.859 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-09-12T13:50:59.860 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2020-09-12T13:50:59.891 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:50:59 +0000 (0:00:01.390) 0:00:30.253 **** 2020-09-12T13:51:17.025 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-09-12T13:51:17.027 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:16.933361", "end": "2020-09-12 13:51:17.004009", "failed_when_result": false, "rc": 0, "start": "2020-09-12 13:51:00.070648", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2020-09-12T13:51:17.061 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:17 +0000 (0:00:17.169) 0:00:47.423 **** 2020-09-12T13:51:17.553 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-09-12T13:51:17.555 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2020-09-12T13:51:17.585 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:17 +0000 (0:00:00.524) 0:00:47.947 **** 2020-09-12T13:51:17.799 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.015287", "end": "2020-09-12 13:51:17.779435", "rc": 0, "start": "2020-09-12 13:51:17.764148", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-12T13:51:17.831 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:17 +0000 (0:00:00.245) 0:00:48.193 **** 2020-09-12T13:51:18.044 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.007771", "end": "2020-09-12 13:51:18.018677", "rc": 0, "start": "2020-09-12 13:51:18.010906", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-12T13:51:18.080 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:18 +0000 (0:00:00.248) 0:00:48.442 **** 2020-09-12T13:51:35.350 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:17.065321", "end": "2020-09-12 13:51:35.323938", "rc": 0, "start": "2020-09-12 13:51:18.258617", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-12T13:51:35.382 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:35 +0000 (0:00:17.302) 0:01:05.744 **** 2020-09-12T13:51:35.588 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-09-12T13:51:35.589 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2020-09-12T13:51:35.619 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:35 +0000 (0:00:00.237) 0:01:05.981 **** 2020-09-12T13:51:35.646 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-09-12T13:51:35.648 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/dnf_systems.yml"], "changed": false} 2020-09-12T13:51:35.679 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:35 +0000 (0:00:00.059) 0:01:06.041 **** 2020-09-12T13:51:35.717 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-09-12T13:51:35.719 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-09-12T13:51:35.757 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:35 +0000 (0:00:00.078) 0:01:06.119 **** 2020-09-12T13:51:35.795 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-09-12T13:51:35.796 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml) => {"ansible_facts": {"common_yum_repos": {"lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "epel_packages": [], "nfs_service": "nfs-server", "ntp_service_name": "chronyd", "packages": ["dnf-utils", "sysstat", "libedit", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse-libs", "openssl", "libuuid", "podman", "podman-docker", "attr", "ant", "lsof", "gettext", "bc", "xfsdump", "blktrace", "usbredir", "libev-devel", "valgrind", "nfs-utils", "ncurses-devel", "gcc", "git", "python3-nose", "python3-virtualenv", "genisoimage", "qemu-img", "qemu-kvm-core", "qemu-kvm-block-rbd", "libacl-devel", "dbench", "autoconf", "gdb", "iozone"], "yum_mirrorlists": ["CentOS-AppStream-mirrorlist", "CentOS-Base-mirrorlist", "CentOS-Extras-mirrorlist"], "yum_repos": {"CentOS-AppStream": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}, "CentOS-Base": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}, "CentOS-Extras": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"} 2020-09-12T13:51:35.830 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:35 +0000 (0:00:00.072) 0:01:06.191 **** 2020-09-12T13:51:35.867 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-09-12T13:51:35.868 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-09-12T13:51:35.904 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:35 +0000 (0:00:00.075) 0:01:06.266 **** 2020-09-12T13:51:36.112 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-09-12T13:51:36.114 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-09-12T13:51:36.150 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:36 +0000 (0:00:00.245) 0:01:06.512 **** 2020-09-12T13:51:36.355 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "gid": 1229, "name": "ubuntu", "state": "present", "system": false} 2020-09-12T13:51:36.386 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:36 +0000 (0:00:00.236) 0:01:06.748 **** 2020-09-12T13:51:36.414 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-09-12T13:51:36.416 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:51:36.449 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:36 +0000 (0:00:00.062) 0:01:06.811 **** 2020-09-12T13:51:36.477 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-09-12T13:51:36.478 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:51:36.509 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:36 +0000 (0:00:00.059) 0:01:06.871 **** 2020-09-12T13:51:36.735 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-09-12T13:51:36.736 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2020-09-12T13:51:36.769 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:36 +0000 (0:00:00.260) 0:01:07.131 **** 2020-09-12T13:51:37.324 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2020-09-12T13:51:37.357 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:37 +0000 (0:00:00.587) 0:01:07.719 **** 2020-09-12T13:51:37.359 INFO:teuthology.task.ansible.out: 2020-09-12T13:51:37.384 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-09-12T13:51:37.386 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi027"}, "changed": false} 2020-09-12T13:51:37.416 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:37 +0000 (0:00:00.058) 0:01:07.778 **** 2020-09-12T13:51:38.399 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-09-12T13:51:38.401 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi027", "ansible_hostname": "smithi027", "ansible_nodename": "smithi027"}, "changed": false, "name": "smithi027"} 2020-09-12T13:51:38.432 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:38 +0000 (0:00:01.015) 0:01:08.794 **** 2020-09-12T13:51:38.928 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-09-12T13:51:38.930 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2020-09-12T13:51:38.971 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:38 +0000 (0:00:00.539) 0:01:09.333 **** 2020-09-12T13:51:39.455 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-09-12T13:51:39.456 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2020-09-12T13:51:39.486 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:39 +0000 (0:00:00.515) 0:01:09.848 **** 2020-09-12T13:51:40.109 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-09-12T13:51:40.111 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-09-12T13:51:40.151 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:40 +0000 (0:00:00.664) 0:01:10.513 **** 2020-09-12T13:51:40.663 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-09-12T13:51:40.665 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2020-09-12T13:51:40.707 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:40 +0000 (0:00:00.556) 0:01:11.069 **** 2020-09-12T13:51:40.926 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2020-09-12T13:51:40.957 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:40 +0000 (0:00:00.250) 0:01:11.319 **** 2020-09-12T13:51:41.173 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "gid": 36, "name": "kvm", "state": "present", "system": false} 2020-09-12T13:51:41.203 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:41 +0000 (0:00:00.246) 0:01:11.565 **** 2020-09-12T13:51:41.431 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-09-12T13:51:41.432 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1229, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-09-12T13:51:41.466 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:41 +0000 (0:00:00.262) 0:01:11.828 **** 2020-09-12T13:51:41.965 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-09-12T13:51:41.967 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2020-09-12T13:51:41.997 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:41 +0000 (0:00:00.530) 0:01:12.359 **** 2020-09-12T13:51:42.506 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2020-09-12T13:51:42.538 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:42 +0000 (0:00:00.540) 0:01:12.900 **** 2020-09-12T13:51:44.461 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-09-12T13:51:44.463 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'. If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:01.713401", "end": "2020-09-12 13:51:44.433763", "rc": 0, "start": "2020-09-12 13:51:42.720362", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-12T13:51:44.493 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:44 +0000 (0:00:01.955) 0:01:14.855 **** 2020-09-12T13:51:45.018 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-09-12T13:51:45.020 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=CentOS-AppStream-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "aa5ad73629677ef17172417e50d2cdadccb7a3f2", "dest": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-AppStream-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 397, "state": "file", "uid": 0} 2020-09-12T13:51:45.493 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=CentOS-Base-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "4ed3c0272e6c4ec99169cd68f0b43d40bdd4b296", "dest": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Base-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 380, "state": "file", "uid": 0} 2020-09-12T13:51:45.958 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=CentOS-Extras-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "a862f43d3b3c7b8b592aa7dce23a6d013ab20bc3", "dest": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Extras-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 382, "state": "file", "uid": 0} 2020-09-12T13:51:45.993 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:45 +0000 (0:00:01.500) 0:01:16.355 **** 2020-09-12T13:51:46.491 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-09-12T13:51:46.493 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/8/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a0e2ac0628a264e14b96b1c57627a3aa344a6c52", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 203, "state": "file", "uid": 0} 2020-09-12T13:51:46.526 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:46 +0000 (0:00:00.532) 0:01:16.888 **** 2020-09-12T13:51:47.055 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** ok: [smithi027.front.sepia.ceph.com] => (item={'key': 'CentOS-AppStream', 'value': {'name': 'CentOS-$releasever - AppStream', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "9b147c879e95cd23f3bbde027da102febcaad1a8", "dest": "/etc/yum.repos.d/CentOS-AppStream.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-AppStream", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 289, "state": "file", "uid": 0} 2020-09-12T13:51:47.528 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'key': 'CentOS-Base', 'value': {'name': 'CentOS-$releasever - Base', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Base-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "77e4590ed0ecec056fa44f4e65af60e110822b85", "dest": "/etc/yum.repos.d/CentOS-Base.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Base", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 274, "state": "file", "uid": 0} 2020-09-12T13:51:47.988 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'key': 'CentOS-Extras', 'value': {'name': 'CentOS-$releasever - Extras', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Extras-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "2856826614c7f1cb9409228f6c41319a023ca754", "dest": "/etc/yum.repos.d/CentOS-Extras.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Extras", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 280, "state": "file", "uid": 0} 2020-09-12T13:51:48.026 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:48 +0000 (0:00:01.497) 0:01:18.386 **** 2020-09-12T13:51:48.051 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-09-12T13:51:48.089 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:48 +0000 (0:00:00.064) 0:01:18.451 **** 2020-09-12T13:51:48.117 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:51:48.150 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:48 +0000 (0:00:00.061) 0:01:18.512 **** 2020-09-12T13:51:48.189 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-09-12T13:51:48.190 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-09-12T13:51:48.202 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-09-12T13:51:48.235 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:48 +0000 (0:00:00.084) 0:01:18.597 **** 2020-09-12T13:51:48.263 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-09-12T13:51:48.265 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:51:48.297 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:48 +0000 (0:00:00.061) 0:01:18.659 **** 2020-09-12T13:51:48.334 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-12T13:51:48.336 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2020-09-12T13:51:48.366 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:48 +0000 (0:00:00.069) 0:01:18.728 **** 2020-09-12T13:51:48.393 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:51:48.425 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:48 +0000 (0:00:00.059) 0:01:18.787 **** 2020-09-12T13:51:48.463 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:51:48.494 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:48 +0000 (0:00:00.068) 0:01:18.856 **** 2020-09-12T13:51:48.712 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* fatal: [smithi027.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.007077", "end": "2020-09-12 13:51:48.689032", "msg": "non-zero return code", "rc": 1, "start": "2020-09-12 13:51:48.681955", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2020-09-12T13:51:48.745 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:48 +0000 (0:00:00.250) 0:01:19.107 **** 2020-09-12T13:51:48.782 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:51:48.814 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:48 +0000 (0:00:00.069) 0:01:19.176 **** 2020-09-12T13:51:51.240 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-09-12T13:51:51.242 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Removed: librados2-1:12.2.7-9.el8.x86_64", "Removed: librbd1-1:12.2.7-9.el8.x86_64"]} 2020-09-12T13:51:51.272 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:51 +0000 (0:00:02.457) 0:01:21.634 **** 2020-09-12T13:51:53.291 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-09-12T13:51:53.292 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: boost-program-options-1.66.0-6.el8.x86_64", "Removed: boost-random-1.66.0-6.el8.x86_64"]} 2020-09-12T13:51:53.324 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:53 +0000 (0:00:02.051) 0:01:23.686 **** 2020-09-12T13:51:56.543 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-09-12T13:51:56.545 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: dnf-utils", "Installed: sysstat", "Installed: libedit", "Installed: boost-thread", "Installed: xfsprogs", "Installed: gdisk", "Installed: parted", "Installed: libgcrypt", "Installed: fuse-libs", "Installed: openssl", "Installed: libuuid", "Installed: podman", "Installed: podman-docker", "Installed: attr", "Installed: ant", "Installed: lsof", "Installed: gettext", "Installed: bc", "Installed: xfsdump", "Installed: blktrace", "Installed: usbredir", "Installed: libev-devel", "Installed: valgrind", "Installed: nfs-utils", "Installed: ncurses-devel", "Installed: gcc", "Installed: git", "Installed: python3-nose", "Installed: python3-virtualenv", "Installed: genisoimage", "Installed: qemu-img", "Installed: qemu-kvm-core", "Installed: qemu-kvm-block-rbd", "Installed: libacl-devel", "Installed: dbench", "Installed: autoconf", "Installed: gdb", "Installed: iozone", "Installed: boost-random-1.66.0-6.el8.x86_64", "Installed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Installed: librados2-1:12.2.7-9.el8.x86_64", "Installed: librbd1-1:12.2.7-9.el8.x86_64", "Installed: boost-program-options-1.66.0-6.el8.x86_64"]} 2020-09-12T13:51:56.576 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:56 +0000 (0:00:03.251) 0:01:26.938 **** 2020-09-12T13:51:56.602 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-09-12T13:51:56.604 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:51:56.636 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:56 +0000 (0:00:00.060) 0:01:26.998 **** 2020-09-12T13:51:57.619 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-09-12T13:51:57.621 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2020-09-12T13:51:57.652 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:57 +0000 (0:00:01.015) 0:01:28.014 **** 2020-09-12T13:51:57.689 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-09-12T13:51:57.690 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:51:57.723 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:57 +0000 (0:00:00.071) 0:01:28.085 **** 2020-09-12T13:51:58.062 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-09-12T13:51:58.063 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "sysinit.target dbus.socket system.slice basic.target dbus.service polkit.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target network-pre.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "iptables.service shutdown.target ipset.service ip6tables.service ebtables.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTim 2020-09-12T13:51:58.065 INFO:teuthology.task.ansible.out:estampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127450", "LimitNPROCSoft": "127450", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127450", 2020-09-12T13:51:58.066 INFO:teuthology.task.ansible.out:"LimitSIGPENDINGSoft": "127450", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "dbus.socket sysinit.target system.slice", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestampMonotonic": "0", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3 2020-09-12T13:51:58.068 INFO:teuthology.task.ansible.out:", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-09-12T13:51:58.096 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:58 +0000 (0:00:00.372) 0:01:28.458 **** 2020-09-12T13:51:58.440 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-09-12T13:51:58.442 INFO:teuthology.task.ansible.out:fatal: [smithi027.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2020-09-12T13:51:58.472 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:58 +0000 (0:00:00.376) 0:01:28.834 **** 2020-09-12T13:51:59.265 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-09-12T13:51:59.267 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2020-09-12T13:51:59.298 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:59 +0000 (0:00:00.825) 0:01:29.660 **** 2020-09-12T13:51:59.324 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2020-09-12T13:51:59.326 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:51:59.359 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:59 +0000 (0:00:00.060) 0:01:29.720 **** 2020-09-12T13:51:59.386 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2020-09-12T13:51:59.387 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:51:59.417 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:59 +0000 (0:00:00.058) 0:01:29.779 **** 2020-09-12T13:51:59.444 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-09-12T13:51:59.446 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:51:59.477 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:59 +0000 (0:00:00.060) 0:01:29.839 **** 2020-09-12T13:51:59.514 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-09-12T13:51:59.516 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:51:59.547 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:59 +0000 (0:00:00.070) 0:01:29.909 **** 2020-09-12T13:51:59.574 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2020-09-12T13:51:59.576 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:51:59.605 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:59 +0000 (0:00:00.057) 0:01:29.967 **** 2020-09-12T13:51:59.631 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-09-12T13:51:59.633 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:51:59.665 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:59 +0000 (0:00:00.059) 0:01:30.027 **** 2020-09-12T13:51:59.691 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-09-12T13:51:59.722 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:59 +0000 (0:00:00.057) 0:01:30.084 **** 2020-09-12T13:51:59.750 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-09-12T13:51:59.752 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:51:59.781 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:59 +0000 (0:00:00.058) 0:01:30.143 **** 2020-09-12T13:51:59.807 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-09-12T13:51:59.809 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:51:59.840 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:59 +0000 (0:00:00.059) 0:01:30.202 **** 2020-09-12T13:51:59.877 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-09-12T13:51:59.879 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-09-12T13:51:59.911 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:59 +0000 (0:00:00.070) 0:01:30.273 **** 2020-09-12T13:51:59.937 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-09-12T13:51:59.939 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:51:59.969 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:51:59 +0000 (0:00:00.058) 0:01:30.331 **** 2020-09-12T13:51:59.995 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-09-12T13:52:00.027 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:00 +0000 (0:00:00.057) 0:01:30.389 **** 2020-09-12T13:52:00.053 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:00.085 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:00 +0000 (0:00:00.058) 0:01:30.447 **** 2020-09-12T13:52:00.113 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:00.144 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:00 +0000 (0:00:00.059) 0:01:30.506 **** 2020-09-12T13:52:00.175 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-09-12T13:52:00.177 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:00.207 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:00 +0000 (0:00:00.062) 0:01:30.569 **** 2020-09-12T13:52:00.233 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-09-12T13:52:00.235 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:00.265 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:00 +0000 (0:00:00.057) 0:01:30.627 **** 2020-09-12T13:52:00.291 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-09-12T13:52:00.293 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:00.322 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:00 +0000 (0:00:00.057) 0:01:30.684 **** 2020-09-12T13:52:00.348 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-09-12T13:52:00.350 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:00.380 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:00 +0000 (0:00:00.058) 0:01:30.742 **** 2020-09-12T13:52:00.406 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-09-12T13:52:00.408 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:00.437 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:00 +0000 (0:00:00.057) 0:01:30.799 **** 2020-09-12T13:52:00.466 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-09-12T13:52:00.467 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:00.497 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:00 +0000 (0:00:00.059) 0:01:30.859 **** 2020-09-12T13:52:00.534 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-09-12T13:52:00.536 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:00.586 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:00 +0000 (0:00:00.088) 0:01:30.948 **** 2020-09-12T13:52:00.615 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-09-12T13:52:00.616 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:00.648 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:00 +0000 (0:00:00.062) 0:01:31.010 **** 2020-09-12T13:52:00.675 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-09-12T13:52:00.678 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:00.707 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:00 +0000 (0:00:00.059) 0:01:31.069 **** 2020-09-12T13:52:00.734 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-09-12T13:52:00.736 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:00.768 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:00 +0000 (0:00:00.060) 0:01:31.130 **** 2020-09-12T13:52:00.795 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-09-12T13:52:00.796 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:00.829 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:00 +0000 (0:00:00.061) 0:01:31.192 **** 2020-09-12T13:52:00.856 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-09-12T13:52:00.858 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:00.896 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:00 +0000 (0:00:00.066) 0:01:31.258 **** 2020-09-12T13:52:00.925 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:00.963 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:00 +0000 (0:00:00.065) 0:01:31.324 **** 2020-09-12T13:52:00.993 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:01.026 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:01 +0000 (0:00:00.064) 0:01:31.388 **** 2020-09-12T13:52:01.053 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:01.087 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:01 +0000 (0:00:00.060) 0:01:31.449 **** 2020-09-12T13:52:01.114 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-09-12T13:52:01.116 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:01.155 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:01 +0000 (0:00:00.068) 0:01:31.517 **** 2020-09-12T13:52:01.182 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-09-12T13:52:01.183 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:01.216 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:01 +0000 (0:00:00.060) 0:01:31.578 **** 2020-09-12T13:52:01.243 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-09-12T13:52:01.245 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:01.275 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:01 +0000 (0:00:00.059) 0:01:31.637 **** 2020-09-12T13:52:01.302 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-09-12T13:52:01.303 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:01.334 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:01 +0000 (0:00:00.059) 0:01:31.696 **** 2020-09-12T13:52:01.361 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-09-12T13:52:01.362 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:01.393 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:01 +0000 (0:00:00.058) 0:01:31.755 **** 2020-09-12T13:52:01.419 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-12T13:52:01.421 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:01.455 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:01 +0000 (0:00:00.062) 0:01:31.817 **** 2020-09-12T13:52:01.482 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-09-12T13:52:01.482 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:01.514 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:01 +0000 (0:00:00.058) 0:01:31.876 **** 2020-09-12T13:52:01.540 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-09-12T13:52:01.542 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:01.573 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:01 +0000 (0:00:00.059) 0:01:31.935 **** 2020-09-12T13:52:01.600 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:01.631 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:01 +0000 (0:00:00.058) 0:01:31.994 **** 2020-09-12T13:52:01.659 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-09-12T13:52:01.660 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:01.691 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:01 +0000 (0:00:00.059) 0:01:32.053 **** 2020-09-12T13:52:01.919 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-09-12T13:52:01.921 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2020-09-12T13:52:01.965 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:01 +0000 (0:00:00.273) 0:01:32.327 **** 2020-09-12T13:52:02.198 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-09-12T13:52:02.228 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:02 +0000 (0:00:00.263) 0:01:32.590 **** 2020-09-12T13:52:02.448 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** [WARNING]: Cannot set fs attributes on a non-existent symlink target. follow should be set to False to avoid this. ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2020-09-12T13:52:02.487 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:02 +0000 (0:00:00.258) 0:01:32.849 **** 2020-09-12T13:52:02.515 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-09-12T13:52:02.517 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:02.549 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:02 +0000 (0:00:00.061) 0:01:32.911 **** 2020-09-12T13:52:02.575 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-09-12T13:52:02.577 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:02.608 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:02 +0000 (0:00:00.059) 0:01:32.970 **** 2020-09-12T13:52:02.635 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-09-12T13:52:02.637 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:02.667 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:02 +0000 (0:00:00.058) 0:01:33.029 **** 2020-09-12T13:52:02.693 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:02.727 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:02 +0000 (0:00:00.059) 0:01:33.089 **** 2020-09-12T13:52:02.765 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-09-12T13:52:02.766 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:02.801 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:02 +0000 (0:00:00.074) 0:01:33.163 **** 2020-09-12T13:52:02.828 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:02.861 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:02 +0000 (0:00:00.059) 0:01:33.223 **** 2020-09-12T13:52:02.889 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:02.922 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:02 +0000 (0:00:00.061) 0:01:33.284 **** 2020-09-12T13:52:02.948 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-09-12T13:52:02.950 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:02.991 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:02 +0000 (0:00:00.068) 0:01:33.353 **** 2020-09-12T13:52:03.021 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-09-12T13:52:03.023 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:03.067 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:03 +0000 (0:00:00.075) 0:01:33.429 **** 2020-09-12T13:52:03.096 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-09-12T13:52:03.098 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:03.129 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:03 +0000 (0:00:00.062) 0:01:33.491 **** 2020-09-12T13:52:03.157 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-09-12T13:52:03.159 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:03.189 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:03 +0000 (0:00:00.060) 0:01:33.551 **** 2020-09-12T13:52:03.216 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-09-12T13:52:03.218 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:03.248 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:03 +0000 (0:00:00.058) 0:01:33.610 **** 2020-09-12T13:52:03.276 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-09-12T13:52:03.277 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:03.311 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:03 +0000 (0:00:00.063) 0:01:33.673 **** 2020-09-12T13:52:03.337 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-09-12T13:52:03.339 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:03.369 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:03 +0000 (0:00:00.057) 0:01:33.731 **** 2020-09-12T13:52:03.395 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-09-12T13:52:03.397 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:03.427 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:03 +0000 (0:00:00.057) 0:01:33.789 **** 2020-09-12T13:52:03.453 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-09-12T13:52:03.455 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:03.485 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:03 +0000 (0:00:00.057) 0:01:33.846 **** 2020-09-12T13:52:03.511 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-09-12T13:52:03.513 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:03.543 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:03 +0000 (0:00:00.058) 0:01:33.905 **** 2020-09-12T13:52:03.570 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-09-12T13:52:03.571 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:03.609 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:03 +0000 (0:00:00.066) 0:01:33.971 **** 2020-09-12T13:52:03.639 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-09-12T13:52:03.640 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:03.671 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:03 +0000 (0:00:00.061) 0:01:34.033 **** 2020-09-12T13:52:03.709 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:03.747 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:03 +0000 (0:00:00.075) 0:01:34.109 **** 2020-09-12T13:52:03.773 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-09-12T13:52:03.775 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:03.806 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:03 +0000 (0:00:00.059) 0:01:34.168 **** 2020-09-12T13:52:03.833 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:03.866 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:03 +0000 (0:00:00.059) 0:01:34.228 **** 2020-09-12T13:52:03.903 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-12T13:52:03.905 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-09-12T13:52:03.935 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:03 +0000 (0:00:00.069) 0:01:34.297 **** 2020-09-12T13:52:03.973 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-09-12T13:52:03.974 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-09-12T13:52:03.984 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-09-12T13:52:03.995 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-09-12T13:52:03.996 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-09-12T13:52:04.006 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-09-12T13:52:04.008 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-09-12T13:52:04.018 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-09-12T13:52:04.060 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:04 +0000 (0:00:00.125) 0:01:34.422 **** 2020-09-12T13:52:04.262 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-09-12T13:52:04.263 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003503", "end": "2020-09-12 13:52:04.236860", "rc": 0, "start": "2020-09-12 13:52:04.233357", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-09-12T13:52:04.297 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:04 +0000 (0:00:00.237) 0:01:34.659 **** 2020-09-12T13:52:04.334 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-09-12T13:52:04.336 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:04.372 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:04 +0000 (0:00:00.074) 0:01:34.734 **** 2020-09-12T13:52:04.403 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-09-12T13:52:04.405 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:04.436 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:04 +0000 (0:00:00.063) 0:01:34.798 **** 2020-09-12T13:52:04.462 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-09-12T13:52:04.464 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:04.494 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:04 +0000 (0:00:00.058) 0:01:34.856 **** 2020-09-12T13:52:04.521 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-09-12T13:52:04.522 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:04.552 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:04 +0000 (0:00:00.057) 0:01:34.914 **** 2020-09-12T13:52:05.533 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-09-12T13:52:05.534 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: gdisk", "Installed: device-mapper"]} 2020-09-12T13:52:05.565 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:05 +0000 (0:00:01.013) 0:01:35.927 **** 2020-09-12T13:52:05.612 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-09-12T13:52:05.614 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930530242560, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227180235, 'block_used': 12914873, 'inode_total': 61054976, 'inode_available': 60990170, 'inode_used': 64806, 'uuid': '7e53389d-f8ef-4bef-b3f0-509d591fc1e4'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227180235, "block_size": 4096, "block_total": 240095108, "block_used": 12914873, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990170, "inode_total": 61054976, "inode_used": 64806, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930530242560, "size_total": 983429562368, "uuid": "7e53389d-f8ef-4bef-b3f0-509d591fc1e4"}} 2020-09-12T13:52:05.645 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:05 +0000 (0:00:00.079) 0:01:36.006 **** 2020-09-12T13:52:05.848 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.005396", "end": "2020-09-12 13:52:05.830459", "rc": 0, "start": "2020-09-12 13:52:05.825063", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-12T13:52:05.885 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:05 +0000 (0:00:00.240) 0:01:36.247 **** 2020-09-12T13:52:05.925 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-09-12T13:52:05.926 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930530242560, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227180235, 'block_used': 12914873, 'inode_total': 61054976, 'inode_available': 60990170, 'inode_used': 64806, 'uuid': '7e53389d-f8ef-4bef-b3f0-509d591fc1e4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227180235, "block_size": 4096, "block_total": 240095108, "block_used": 12914873, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990170, "inode_total": 61054976, "inode_used": 64806, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930530242560, "size_total": 983429562368, "uuid": "7e53389d-f8ef-4bef-b3f0-509d591fc1e4"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:05.956 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:05 +0000 (0:00:00.071) 0:01:36.318 **** 2020-09-12T13:52:05.983 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-09-12T13:52:06.015 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:06 +0000 (0:00:00.058) 0:01:36.377 **** 2020-09-12T13:52:07.595 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-09-12T13:52:07.597 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-x8uBHDTOUh29d84d9AZ2Lfl5nEL73P2Pp1ebs9HsL9XIbCbGpLSrQFkZdRlQ9yfB'], 'uuids': ['2f3204ff-b0d5-4a31-b42e-6025ff7e3940'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.353633", "end": "2020-09-12 13:52:07.566285", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-x8uBHDTOUh29d84d9AZ2Lfl5nEL73P2Pp1ebs9HsL9XIbCbGpLSrQFkZdRlQ9yfB"], "labels": [], "masters": [], "uuids": ["2f3204ff-b0d5-4a31-b42e-6025ff7e3940"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-12 13:52:06.212652", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-09-12T13:52:08.877 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-zBMvPM-PY4U-zWj0-gwey-7aNk-CHdU-LVdOnZ', 'nvme-INTEL_SSDPEDMD400G4_CVFT5331001C400BGN', 'nvme-nvme.8086-43564654353333313030314334303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.077278", "end": "2020-09-12 13:52:08.844612", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-zBMvPM-PY4U-zWj0-gwey-7aNk-CHdU-LVdOnZ", "nvme-INTEL_SSDPEDMD400G4_CVFT5331001C400BGN", "nvme-nvme.8086-43564654353333313030314334303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-12 13:52:07.767334", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-09-12T13:52:08.879 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-09-12T13:52:10.126 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-x8uBHDTOUh29d84d9AZ2Lfl5nEL73P2PFb7wzom0rdpvw0pe84EAnUX3Tld3Sw1f'], 'uuids': ['2ecd1ec0-cd55-4655-83b1-842635414e45'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.045142", "end": "2020-09-12 13:52:10.098903", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-x8uBHDTOUh29d84d9AZ2Lfl5nEL73P2PFb7wzom0rdpvw0pe84EAnUX3Tld3Sw1f"], "labels": [], "masters": [], "uuids": ["2ecd1ec0-cd55-4655-83b1-842635414e45"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-12 13:52:09.053761", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-09-12T13:52:11.381 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-x8uBHDTOUh29d84d9AZ2Lfl5nEL73P2PpCiCOHOqwY3n4f9UTELO8hQkrakIdVeg'], 'uuids': ['041c4b45-8db8-4809-b1fb-2c9fd15823c6'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.057169", "end": "2020-09-12 13:52:11.356862", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-x8uBHDTOUh29d84d9AZ2Lfl5nEL73P2PpCiCOHOqwY3n4f9UTELO8hQkrakIdVeg"], "labels": [], "masters": [], "uuids": ["041c4b45-8db8-4809-b1fb-2c9fd15823c6"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-12 13:52:10.299693", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-09-12T13:52:12.636 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-x8uBHDTOUh29d84d9AZ2Lfl5nEL73P2P7cb3bBbll42N2YHVA6n1xSOo72D3GYxb'], 'uuids': ['859a61dc-33a2-4d69-987e-f649fc7abdb4'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.048216", "end": "2020-09-12 13:52:12.607639", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-x8uBHDTOUh29d84d9AZ2Lfl5nEL73P2P7cb3bBbll42N2YHVA6n1xSOo72D3GYxb"], "labels": [], "masters": [], "uuids": ["859a61dc-33a2-4d69-987e-f649fc7abdb4"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-12 13:52:11.559423", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-09-12T13:52:12.648 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4H3N7', 'wwn-0x5000c500871b45cd'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W4H3N7', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c500871b45cd', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4H3N7-part1', 'wwn-0x5000c500871b45cd-part1'], 'uuids': ['7e53389d-f8ef-4bef-b3f0-509d591fc1e4'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '7e53389d-f8ef-4bef-b3f0-509d591fc1e4', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4H3N7", "wwn-0x5000c500871b45cd"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4H3N7-part1", "wwn-0x5000c500871b45cd-part1"], "labels": [], "masters": [], "uuids": ["7e53389d-f8ef-4bef-b3f0-509d591fc1e4"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "7e53389d-f8ef-4bef-b3f0-509d591fc1e4"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4H3N7", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871b45cd"}}, 2020-09-12T13:52:12.649 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-09-12T13:52:13.925 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-x8uBHDTOUh29d84d9AZ2Lfl5nEL73P2PPz7ifQRfS2IBhkLKPL9vuhvSgLENULZ4'], 'uuids': ['191a728c-9617-4e27-829b-e13c4cd1d9b5'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.066659", "end": "2020-09-12 13:52:13.890970", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-x8uBHDTOUh29d84d9AZ2Lfl5nEL73P2PPz7ifQRfS2IBhkLKPL9vuhvSgLENULZ4"], "labels": [], "masters": [], "uuids": ["191a728c-9617-4e27-829b-e13c4cd1d9b5"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-12 13:52:12.824311", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-09-12T13:52:13.973 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:13 +0000 (0:00:07.958) 0:01:44.335 **** 2020-09-12T13:52:14.372 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-09-12T13:52:14.374 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-x8uBHDTOUh29d84d9AZ2Lfl5nEL73P2Pp1ebs9HsL9XIbCbGpLSrQFkZdRlQ9yfB'], 'uuids': ['2f3204ff-b0d5-4a31-b42e-6025ff7e3940'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.145464", "end": "2020-09-12 13:52:14.346705", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-x8uBHDTOUh29d84d9AZ2Lfl5nEL73P2Pp1ebs9HsL9XIbCbGpLSrQFkZdRlQ9yfB"], "labels": [], "masters": [], "uuids": ["2f3204ff-b0d5-4a31-b42e-6025ff7e3940"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-12 13:52:14.201241", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.142777 s, 808 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.142777 s, 808 MB/s"], "stdout": "", "stdout_lines": []} 2020-09-12T13:52:14.606 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-zBMvPM-PY4U-zWj0-gwey-7aNk-CHdU-LVdOnZ', 'nvme-INTEL_SSDPEDMD400G4_CVFT5331001C400BGN', 'nvme-nvme.8086-43564654353333313030314334303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.036053", "end": "2020-09-12 13:52:14.574980", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-zBMvPM-PY4U-zWj0-gwey-7aNk-CHdU-LVdOnZ", "nvme-INTEL_SSDPEDMD400G4_CVFT5331001C400BGN", "nvme-nvme.8086-43564654353333313030314334303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-12 13:52:14.538927", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0334813 s, 3.4 GB/s", "std 2020-09-12T13:52:14.608 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0334813 s, 3.4 GB/s"], "stdout": "", "stdout_lines": []} 2020-09-12T13:52:14.942 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-x8uBHDTOUh29d84d9AZ2Lfl5nEL73P2PFb7wzom0rdpvw0pe84EAnUX3Tld3Sw1f'], 'uuids': ['2ecd1ec0-cd55-4655-83b1-842635414e45'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.142321", "end": "2020-09-12 13:52:14.913523", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-x8uBHDTOUh29d84d9AZ2Lfl5nEL73P2PFb7wzom0rdpvw0pe84EAnUX3Tld3Sw1f"], "labels": [], "masters": [], "uuids": ["2ecd1ec0-cd55-4655-83b1-842635414e45"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-12 13:52:14.771202", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.139679 s, 826 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.139679 s, 826 MB/s"], "stdout": "", "stdout_lines": []} 2020-09-12T13:52:15.279 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-x8uBHDTOUh29d84d9AZ2Lfl5nEL73P2PpCiCOHOqwY3n4f9UTELO8hQkrakIdVeg'], 'uuids': ['041c4b45-8db8-4809-b1fb-2c9fd15823c6'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.136663", "end": "2020-09-12 13:52:15.251432", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-x8uBHDTOUh29d84d9AZ2Lfl5nEL73P2PpCiCOHOqwY3n4f9UTELO8hQkrakIdVeg"], "labels": [], "masters": [], "uuids": ["041c4b45-8db8-4809-b1fb-2c9fd15823c6"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-12 13:52:15.114769", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.13401 s, 861 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.13401 s, 861 MB/s"], "stdout": "", "stdout_lines": []} 2020-09-12T13:52:15.613 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-x8uBHDTOUh29d84d9AZ2Lfl5nEL73P2P7cb3bBbll42N2YHVA6n1xSOo72D3GYxb'], 'uuids': ['859a61dc-33a2-4d69-987e-f649fc7abdb4'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.136288", "end": "2020-09-12 13:52:15.583887", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-x8uBHDTOUh29d84d9AZ2Lfl5nEL73P2P7cb3bBbll42N2YHVA6n1xSOo72D3GYxb"], "labels": [], "masters": [], "uuids": ["859a61dc-33a2-4d69-987e-f649fc7abdb4"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-12 13:52:15.447599", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.133735 s, 862 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.133735 s, 862 MB/s"], "stdout": "", "stdout_lines": []} 2020-09-12T13:52:15.614 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4H3N7', 'wwn-0x5000c500871b45cd'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W4H3N7', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c500871b45cd', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4H3N7-part1', 'wwn-0x5000c500871b45cd-part1'], 'uuids': ['7e53389d-f8ef-4bef-b3f0-509d591fc1e4'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '7e53389d-f8ef-4bef-b3f0-509d591fc1e4', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4H3N7", "wwn-0x5000c500871b45cd"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4H3N7-part1", "wwn-0x5000c500871b45cd-part1"], "labels": [], "masters": [], "uuids": ["7e53389d-f8ef-4bef-b3f0-509d591fc1e4"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "7e53389d-f8ef-4bef-b3f0-509d591fc1e4"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4H3N7", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871b45cd"}}, 2020-09-12T13:52:15.616 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-09-12T13:52:15.947 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-x8uBHDTOUh29d84d9AZ2Lfl5nEL73P2PPz7ifQRfS2IBhkLKPL9vuhvSgLENULZ4'], 'uuids': ['191a728c-9617-4e27-829b-e13c4cd1d9b5'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.136483", "end": "2020-09-12 13:52:15.923239", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-x8uBHDTOUh29d84d9AZ2Lfl5nEL73P2PPz7ifQRfS2IBhkLKPL9vuhvSgLENULZ4"], "labels": [], "masters": [], "uuids": ["191a728c-9617-4e27-829b-e13c4cd1d9b5"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-12 13:52:15.786756", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.133874 s, 862 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.133874 s, 862 MB/s"], "stdout": "", "stdout_lines": []} 2020-09-12T13:52:15.984 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:15 +0000 (0:00:02.011) 0:01:46.346 **** 2020-09-12T13:52:16.273 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-09-12T13:52:16.275 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.080344", "end": "2020-09-12 13:52:16.244380", "rc": 0, "start": "2020-09-12 13:52:16.164036", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-12T13:52:16.318 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:16 +0000 (0:00:00.333) 0:01:46.680 **** 2020-09-12T13:52:16.565 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-09-12T13:52:16.567 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.042710", "end": "2020-09-12 13:52:16.542979", "rc": 0, "start": "2020-09-12 13:52:16.500269", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-12T13:52:16.599 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:16 +0000 (0:00:00.280) 0:01:46.960 **** 2020-09-12T13:52:16.625 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-09-12T13:52:16.659 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:16 +0000 (0:00:00.061) 0:01:47.021 **** 2020-09-12T13:52:16.686 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-09-12T13:52:16.688 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:16.720 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:16 +0000 (0:00:00.060) 0:01:47.082 **** 2020-09-12T13:52:16.746 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-09-12T13:52:16.748 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:16.778 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:16 +0000 (0:00:00.058) 0:01:47.140 **** 2020-09-12T13:52:16.806 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-09-12T13:52:16.808 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:16.843 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:16 +0000 (0:00:00.065) 0:01:47.205 **** 2020-09-12T13:52:16.870 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-09-12T13:52:16.872 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:16.904 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:16 +0000 (0:00:00.060) 0:01:47.266 **** 2020-09-12T13:52:17.109 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ ok: [smithi027.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-09-12T13:52:17.141 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:17 +0000 (0:00:00.237) 0:01:47.503 **** 2020-09-12T13:52:17.178 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-09-12T13:52:17.180 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930530242560, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227180235, 'block_used': 12914873, 'inode_total': 61054976, 'inode_available': 60990170, 'inode_used': 64806, 'uuid': '7e53389d-f8ef-4bef-b3f0-509d591fc1e4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227180235, "block_size": 4096, "block_total": 240095108, "block_used": 12914873, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990170, "inode_total": 61054976, "inode_used": 64806, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930530242560, "size_total": 983429562368, "uuid": "7e53389d-f8ef-4bef-b3f0-509d591fc1e4"}, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:17.211 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:17 +0000 (0:00:00.069) 0:01:47.573 **** 2020-09-12T13:52:17.238 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-09-12T13:52:17.240 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:17.270 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:17 +0000 (0:00:00.059) 0:01:47.632 **** 2020-09-12T13:52:17.307 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-12T13:52:17.309 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:17.341 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:17 +0000 (0:00:00.070) 0:01:47.703 **** 2020-09-12T13:52:17.369 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-12T13:52:17.371 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:17.406 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:17 +0000 (0:00:00.064) 0:01:47.768 **** 2020-09-12T13:52:17.432 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-09-12T13:52:17.434 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:17.464 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:17 +0000 (0:00:00.057) 0:01:47.826 **** 2020-09-12T13:52:17.490 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-09-12T13:52:17.492 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:17.521 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:17 +0000 (0:00:00.057) 0:01:47.883 **** 2020-09-12T13:52:17.548 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-09-12T13:52:17.549 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:17.579 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:17 +0000 (0:00:00.057) 0:01:47.941 **** 2020-09-12T13:52:18.295 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-09-12T13:52:18.297 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-09-12T13:52:18.328 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:18 +0000 (0:00:00.748) 0:01:48.690 **** 2020-09-12T13:52:18.953 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-09-12T13:52:18.955 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-09-12T13:52:19.296 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-09-12T13:52:19.628 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-09-12T13:52:19.980 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-09-12T13:52:20.302 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-09-12T13:52:20.336 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:20 +0000 (0:00:02.008) 0:01:50.698 **** 2020-09-12T13:52:20.552 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-09-12T13:52:20.587 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:20 +0000 (0:00:00.251) 0:01:50.949 **** 2020-09-12T13:52:20.815 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-09-12T13:52:20.817 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-09-12T13:52:21.005 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-09-12T13:52:21.195 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-09-12T13:52:21.386 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-09-12T13:52:21.409 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:21.456 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:21 +0000 (0:00:00.869) 0:01:51.818 **** 2020-09-12T13:52:21.673 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-09-12T13:52:21.675 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-09-12T13:52:21.711 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:21 +0000 (0:00:00.254) 0:01:52.073 **** 2020-09-12T13:52:21.749 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-09-12T13:52:21.751 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-09-12T13:52:21.787 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:21 +0000 (0:00:00.076) 0:01:52.149 **** 2020-09-12T13:52:22.351 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-09-12T13:52:22.353 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true} 2020-09-12T13:52:22.391 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:22 +0000 (0:00:00.603) 0:01:52.753 **** 2020-09-12T13:52:22.973 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-09-12T13:52:22.974 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-09-12T13:52:23.009 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:23 +0000 (0:00:00.618) 0:01:53.371 **** 2020-09-12T13:52:23.041 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-09-12T13:52:23.043 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:23.079 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:23 +0000 (0:00:00.069) 0:01:53.441 **** 2020-09-12T13:52:23.106 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-09-12T13:52:23.108 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:23.138 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:23 +0000 (0:00:00.059) 0:01:53.500 **** 2020-09-12T13:52:23.164 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-09-12T13:52:23.166 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:23.196 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:23 +0000 (0:00:00.057) 0:01:53.558 **** 2020-09-12T13:52:23.222 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-09-12T13:52:23.224 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:23.258 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:23 +0000 (0:00:00.062) 0:01:53.620 **** 2020-09-12T13:52:23.286 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-09-12T13:52:23.288 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:23.326 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:23 +0000 (0:00:00.067) 0:01:53.688 **** 2020-09-12T13:52:23.353 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-09-12T13:52:23.355 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:23.387 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:23 +0000 (0:00:00.060) 0:01:53.749 **** 2020-09-12T13:52:23.414 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-09-12T13:52:23.416 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:23.445 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:23 +0000 (0:00:00.058) 0:01:53.808 **** 2020-09-12T13:52:23.474 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-09-12T13:52:23.476 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:23.510 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:23 +0000 (0:00:00.063) 0:01:53.871 **** 2020-09-12T13:52:23.538 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:23.574 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:23 +0000 (0:00:00.064) 0:01:53.936 **** 2020-09-12T13:52:23.603 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-09-12T13:52:23.604 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:23.645 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:23 +0000 (0:00:00.070) 0:01:54.006 **** 2020-09-12T13:52:23.671 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-09-12T13:52:23.672 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:23.702 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:23 +0000 (0:00:00.057) 0:01:54.064 **** 2020-09-12T13:52:24.218 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-09-12T13:52:24.220 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "a7ecef00073feab34f868065d34e9dac55134241", "dest": "/etc/chrony.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/chrony.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 215, "state": "file", "uid": 0} 2020-09-12T13:52:24.255 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:24 +0000 (0:00:00.552) 0:01:54.617 **** 2020-09-12T13:52:24.578 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-09-12T13:52:24.579 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "chronyd", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-09-12 13:49:54 UTC", "ActiveEnterTimestampMonotonic": "16002043", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "tmp.mount systemd-tmpfiles-setup.service sysinit.target systemd-journald.socket -.mount ntpdate.service system.slice basic.target ntpd.service sntp.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-09-12 13:49:52 UTC", "AssertTimestampMonotonic": "14517754", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-09-12 13:49:52 UTC", "ConditionTimestampMonotonic": "14517729", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service ntpd.service", "ControlGroup": "/system.slice/chronyd.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description" 2020-09-12T13:52:24.581 INFO:teuthology.task.ansible.out:: "NTP client/server", "DevicePolicy": "auto", "Documentation": "man:chronyd(8) man:chrony.conf(5)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/chronyd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1587", "ExecMainStartTimestamp": "Sat 2020-09-12 13:49:54 UTC", "ExecMainStartTimestampMonotonic": "15853802", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/chronyd ; argv[]=/usr/sbin/chronyd $OPTIONS ; ignore_errors=no ; start_time=[Sat 2020-09-12 13:49:52 UTC] ; stop_time=[Sat 2020-09-12 13:49:54 UTC] ; pid=1569 ; code=exited ; status=0 }", "ExecStartPost": "{ path=/usr/libexec/chrony-helper ; argv[]=/usr/libexec/chrony-helper update-daemon ; ignore_errors=no ; start_time=[Sat 2020-09-12 13:49:54 UTC] ; stop_time=[Sat 2020-09-12 13:49:54 UTC] ; pid=2284 ; code=exited ; status=0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/chronyd.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "chronyd.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-09-12 13:49:52 UTC", "InactiveExitTimestampMonotonic": "14578295", "InvocationID": "4e5b44426bc64970ac3c6c53bf447f4f", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZE 2020-09-12T13:52:24.582 INFO:teuthology.task.ansible.out:Soft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127450", "LimitNPROCSoft": "127450", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127450", "LimitSIGPENDINGSoft": "127450", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1587", "MemoryAccounting": "yes", "MemoryCurrent": "2785280", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "chronyd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PIDFile": "/run/chrony/chronyd.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "yes", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "full", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no" 2020-09-12T13:52:24.584 INFO:teuthology.task.ansible.out:, "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-09-12 13:49:54 UTC", "StateChangeTimestampMonotonic": "16002043", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-09-12 13:49:54 UTC", "WatchdogTimestampMonotonic": "15853817", "WatchdogUSec": "0"}} 2020-09-12T13:52:24.616 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:24 +0000 (0:00:00.360) 0:01:54.978 **** 2020-09-12T13:52:24.643 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-12T13:52:24.645 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-09-12T13:52:24.681 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:24 +0000 (0:00:00.065) 0:01:55.043 **** 2020-09-12T13:52:24.709 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-12T13:52:24.711 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2020-09-12T13:52:24.742 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:24 +0000 (0:00:00.061) 0:01:55.104 **** 2020-09-12T13:52:24.771 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-09-12T13:52:24.773 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:24.804 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:24 +0000 (0:00:00.061) 0:01:55.166 **** 2020-09-12T13:52:24.831 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-09-12T13:52:24.832 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:24.863 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:24 +0000 (0:00:00.059) 0:01:55.225 **** 2020-09-12T13:52:24.890 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-09-12T13:52:24.892 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:24.922 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:24 +0000 (0:00:00.059) 0:01:55.284 **** 2020-09-12T13:52:25.138 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "gid": 1229, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2020-09-12T13:52:25.171 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:25 +0000 (0:00:00.248) 0:01:55.533 **** 2020-09-12T13:52:25.769 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-09-12T13:52:25.771 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1229, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2020-09-12T13:52:25.803 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:25 +0000 (0:00:00.632) 0:01:56.165 **** 2020-09-12T13:52:27.168 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-09-12T13:52:27.170 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages\nRequirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages", "Requirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-09-12T13:52:27.199 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:27 +0000 (0:00:01.395) 0:01:57.561 **** 2020-09-12T13:52:27.227 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-09-12T13:52:27.228 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:27.260 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:27 +0000 (0:00:00.060) 0:01:57.622 **** 2020-09-12T13:52:27.286 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-09-12T13:52:27.288 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"interface": "ens1f0"}, "changed": false} 2020-09-12T13:52:27.329 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:27 +0000 (0:00:00.069) 0:01:57.691 **** 2020-09-12T13:52:27.525 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-09-12T13:52:27.527 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-09-12T13:52:27.556 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:27 +0000 (0:00:00.226) 0:01:57.918 **** 2020-09-12T13:52:27.583 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-09-12T13:52:27.584 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:27.614 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:27 +0000 (0:00:00.058) 0:01:57.976 **** 2020-09-12T13:52:27.641 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-09-12T13:52:27.642 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-09-12T13:52:27.672 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:27 +0000 (0:00:00.057) 0:01:58.034 **** 2020-09-12T13:52:27.699 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-09-12T13:52:27.700 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:27.730 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:27 +0000 (0:00:00.058) 0:01:58.092 **** 2020-09-12T13:52:27.944 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-09-12T13:52:27.946 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-09-12T13:52:27.990 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:27 +0000 (0:00:00.259) 0:01:58.352 **** 2020-09-12T13:52:28.194 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* changed: [smithi027.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-09-12T13:52:28.228 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:28 +0000 (0:00:00.237) 0:01:58.590 **** 2020-09-12T13:52:28.254 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-09-12T13:52:28.256 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:28.286 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:28 +0000 (0:00:00.057) 0:01:58.648 **** 2020-09-12T13:52:28.324 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-09-12T13:52:28.325 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-09-12T13:52:28.336 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=ens1f0) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False"} 2020-09-12T13:52:28.338 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item=ens1f1) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False"} 2020-09-12T13:52:28.375 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:28 +0000 (0:00:00.089) 0:01:58.737 **** 2020-09-12T13:52:28.414 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-09-12T13:52:28.418 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:28.421 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:28.431 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:28.432 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:28.463 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:28 +0000 (0:00:00.087) 0:01:58.825 **** 2020-09-12T13:52:28.489 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:28.523 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:28 +0000 (0:00:00.060) 0:01:58.885 **** 2020-09-12T13:52:28.552 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:28.589 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:28 +0000 (0:00:00.064) 0:01:58.950 **** 2020-09-12T13:52:28.616 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-09-12T13:52:28.617 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:28.649 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:28 +0000 (0:00:00.061) 0:01:59.011 **** 2020-09-12T13:52:28.676 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-09-12T13:52:28.678 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:28.709 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:28 +0000 (0:00:00.059) 0:01:59.071 **** 2020-09-12T13:52:28.735 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-09-12T13:52:28.737 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:28.768 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:28 +0000 (0:00:00.059) 0:01:59.130 **** 2020-09-12T13:52:28.795 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-09-12T13:52:28.797 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:28.828 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:28 +0000 (0:00:00.059) 0:01:59.190 **** 2020-09-12T13:52:28.854 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-09-12T13:52:28.856 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:28.888 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:28 +0000 (0:00:00.059) 0:01:59.250 **** 2020-09-12T13:52:28.914 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-09-12T13:52:28.916 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-12T13:52:28.945 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:28 +0000 (0:00:00.057) 0:01:59.307 **** 2020-09-12T13:52:30.182 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-09-12T13:52:30.184 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:01.006528", "end": "2020-09-12 13:52:30.141354", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-09-12 13:52:29.134826", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-12T13:52:30.380 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.003280", "end": "2020-09-12 13:52:30.349307", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-09-12 13:52:30.346027", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-12T13:52:30.561 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.002793", "end": "2020-09-12 13:52:30.534493", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-09-12 13:52:30.531700", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2020-09-12T13:52:31.781 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:01.021107", "end": "2020-09-12 13:52:31.750819", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-09-12 13:52:30.729712", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-12T13:52:31.816 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:31 +0000 (0:00:02.870) 0:02:02.178 **** 2020-09-12T13:52:31.843 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-09-12T13:52:31.876 INFO:teuthology.task.ansible.out:Saturday 12 September 2020 13:52:31 +0000 (0:00:00.060) 0:02:02.238 **** 2020-09-12T13:52:32.092 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-09-12T13:52:32.094 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2020-09-12T13:52:32.188 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** 2020-09-12T13:52:32.189 INFO:teuthology.task.ansible.out:skipping: no hosts matched 2020-09-12T13:52:32.194 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** 2020-09-12T13:52:32.195 INFO:teuthology.task.ansible.out:skipping: no hosts matched 2020-09-12T13:52:32.199 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched PLAY RECAP ********************************************************************* smithi027.front.sepia.ceph.com : ok=111 changed=31 unreachable=0 failed=0 skipped=178 rescued=0 ignored=3 Saturday 12 September 2020 13:52:32 +0000 (0:00:00.323) 0:02:02.561 **** =============================================================================== 2020-09-12T13:52:32.200 INFO:teuthology.task.ansible.out:common : nrpe - Load SELinux policy package ---------------------------- 17.30s 2020-09-12T13:52:32.202 INFO:teuthology.task.ansible.out:common : nrpe - Remove SELinux policy package -------------------------- 17.17s testnode : Zap all non-root disks --------------------------------------- 7.96s testnode : Install packages --------------------------------------------- 3.25s testnode : List any leftover Ceph artifacts from previous jobs ---------- 2.87s Gathering Facts --------------------------------------------------------- 2.72s testnode : Ensure ceph packages are not present. ------------------------ 2.46s testnode : Ensure ceph dependency packages are not present. ------------- 2.05s testnode : Blow away lingering OSD data and FSIDs ----------------------- 2.01s testnode : Create logical volume(s) ------------------------------------- 2.01s testnode : Rebuild rpmdb ------------------------------------------------ 1.96s testnode : Configure local mirrorlists ---------------------------------- 1.50s testnode : Configure version specific repos in /etc/yum.repos.d/ -------- 1.50s common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/. --- 1.49s common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/. --- 1.42s testnode : Install packages via pip ------------------------------------- 1.40s common : nrpe - Set SELinux boolean nagios_run_sudo true ---------------- 1.39s common : Install Kerberos Packages (RedHat) ----------------------------- 1.25s ansible-managed : Create the cephlab_sudo sudoers.d file. --------------- 1.04s common : nrpe - Install SELinux dependencies ---------------------------- 1.03s 2020-09-12T13:52:32.559 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi027.front.sepia.ceph.com')] 2020-09-12T13:52:32.560 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-09-12T13:52:32.561 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-12T13:52:32.632 INFO:teuthology.orchestra.run.smithi027:> true 2020-09-12T13:52:32.717 INFO:teuthology.run_tasks:Running task clock... 2020-09-12T13:52:32.731 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2020-09-12T13:52:32.731 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-09-12T13:52:32.732 INFO:teuthology.orchestra.run.smithi027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-09-12T13:52:32.795 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2020-09-12T13:52:32.816 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2020-09-12T13:52:32.861 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: ntpd: command not found 2020-09-12T13:52:32.879 INFO:teuthology.orchestra.run.smithi027.stdout:506 Cannot talk to daemon 2020-09-12T13:52:32.899 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntp.service: Unit ntp.service not found. 2020-09-12T13:52:32.920 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2020-09-12T13:52:32.980 INFO:teuthology.orchestra.run.smithi027.stderr:bash: ntpq: command not found 2020-09-12T13:52:32.983 INFO:teuthology.orchestra.run.smithi027.stdout:210 Number of sources = 4 2020-09-12T13:52:32.983 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-09-12T13:52:32.983 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2020-09-12T13:52:32.983 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-09-12T13:52:32.983 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-09-12T13:52:32.984 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-09-12T13:52:32.984 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-09-12T13:52:32.984 INFO:teuthology.run_tasks:Running task install... 2020-09-12T13:52:32.994 DEBUG:teuthology.task.install:project ceph 2020-09-12T13:52:32.994 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': 'f346cc73d15a568d3bec1c99a58953de76f57b85'}} 2020-09-12T13:52:32.995 DEBUG:teuthology.task.install:config {'sha1': 'f346cc73d15a568d3bec1c99a58953de76f57b85'} 2020-09-12T13:52:32.995 INFO:teuthology.task.install:Using flavor: basic 2020-09-12T13:52:33.001 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2020-09-12T13:52:33.001 INFO:teuthology.task.install:extra packages: [] 2020-09-12T13:52:33.001 INFO:teuthology.orchestra.run.smithi027:> sudo lsb_release -is 2020-09-12T13:52:33.063 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS 2020-09-12T13:52:33.064 DEBUG:teuthology.misc:System to be installed: CentOS 2020-09-12T13:52:33.064 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'basic', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f346cc73d15a568d3bec1c99a58953de76f57b85', 'tag': None, 'wait_for_package': False} 2020-09-12T13:52:33.065 INFO:teuthology.orchestra.run.smithi027:> sudo lsb_release -is 2020-09-12T13:52:33.144 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS 2020-09-12T13:52:33.145 DEBUG:teuthology.misc:System to be installed: CentOS 2020-09-12T13:52:33.145 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f346cc73d15a568d3bec1c99a58953de76f57b85 2020-09-12T13:52:33.312 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-09-12-2016/f346cc73d15a568d3bec1c99a58953de76f57b85/centos/8/flavors/default/ 2020-09-12T13:52:33.313 INFO:teuthology.task.install.rpm:Package version is 16.0.0-5485.gf346cc73d15 2020-09-12T13:52:33.661 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-09-12-2016/f346cc73d15a568d3bec1c99a58953de76f57b85/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-09-12-2016/f346cc73d15a568d3bec1c99a58953de76f57b85/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-09-12-2016/f346cc73d15a568d3bec1c99a58953de76f57b85/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2020-09-12T13:52:33.662 INFO:teuthology.orchestra.run.smithi027:> set -ex 2020-09-12T13:52:33.663 INFO:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/yum.repos.d/ceph.repo 2020-09-12T13:52:33.677 INFO:teuthology.orchestra.run.smithi027.stderr:+ sudo dd of=/etc/yum.repos.d/ceph.repo 2020-09-12T13:52:33.694 INFO:teuthology.orchestra.run.smithi027.stderr:1+1 records in 2020-09-12T13:52:33.694 INFO:teuthology.orchestra.run.smithi027.stderr:1+1 records out 2020-09-12T13:52:33.695 INFO:teuthology.orchestra.run.smithi027.stderr:664 bytes copied, 4.3244e-05 s, 15.4 MB/s 2020-09-12T13:52:33.695 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2020-09-12T13:52:33.696 INFO:teuthology.orchestra.run.smithi027:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f346cc73d15a568d3bec1c99a58953de76f57b85/;g' /etc/yum.repos.d/ceph.repo ; fi 2020-09-12T13:52:33.768 INFO:teuthology.orchestra.run.smithi027:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2020-09-12T13:52:33.858 INFO:teuthology.orchestra.run.smithi027:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2020-09-12T13:52:33.888 INFO:teuthology.orchestra.run.smithi027.stdout:check_obsoletes = 1 2020-09-12T13:52:33.889 INFO:teuthology.orchestra.run.smithi027:> sudo yum clean all 2020-09-12T13:52:34.185 INFO:teuthology.orchestra.run.smithi027.stdout:41 files removed 2020-09-12T13:52:34.214 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-radosgw 2020-09-12T13:52:35.371 INFO:teuthology.orchestra.run.smithi027.stdout:ceph packages for x86_64 92 kB/s | 72 kB 00:00 2020-09-12T13:52:36.043 INFO:teuthology.orchestra.run.smithi027.stdout:ceph noarch packages 23 kB/s | 14 kB 00:00 2020-09-12T13:52:36.719 INFO:teuthology.orchestra.run.smithi027.stdout:ceph source packages 2.2 kB/s | 1.4 kB 00:00 2020-09-12T13:52:36.904 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-8 - AppStream 43 MB/s | 7.0 MB 00:00 2020-09-12T13:52:38.171 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-8 - Base 25 MB/s | 2.2 MB 00:00 2020-09-12T13:52:38.617 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-8 - Extras 84 kB/s | 5.5 kB 00:00 2020-09-12T13:52:39.005 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 24 MB/s | 8.0 MB 00:00 2020-09-12T13:52:40.717 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 4.2 MB/s | 21 kB 00:00 2020-09-12T13:52:41.770 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:52:41.782 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:52:41.783 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2020-09-12T13:52:41.783 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:52:41.783 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2020-09-12T13:52:41.783 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 12 M 2020-09-12T13:52:41.783 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2020-09-12T13:52:41.783 INFO:teuthology.orchestra.run.smithi027.stdout: librados2 x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 3.7 M 2020-09-12T13:52:41.784 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1 x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 2.2 M 2020-09-12T13:52:41.784 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2020-09-12T13:52:41.784 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 6.1 M 2020-09-12T13:52:41.784 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 23 M 2020-09-12T13:52:41.784 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 25 k 2020-09-12T13:52:41.784 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2 x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 553 k 2020-09-12T13:52:41.784 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1 x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 485 k 2020-09-12T13:52:41.784 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2 x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 3.6 M 2020-09-12T13:52:41.785 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse 2020-09-12T13:52:41.785 INFO:teuthology.orchestra.run.smithi027.stdout: x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 40 k 2020-09-12T13:52:41.785 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common 2020-09-12T13:52:41.785 INFO:teuthology.orchestra.run.smithi027.stdout: x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 72 k 2020-09-12T13:52:41.785 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 204 k 2020-09-12T13:52:41.785 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 359 k 2020-09-12T13:52:41.786 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 335 k 2020-09-12T13:52:41.786 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 113 k 2020-09-12T13:52:41.786 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka x86_64 0.11.4-1.el8 CentOS-AppStream 353 k 2020-09-12T13:52:41.786 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2020-09-12T13:52:41.786 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable 2020-09-12T13:52:41.787 INFO:teuthology.orchestra.run.smithi027.stdout: noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2020-09-12T13:52:41.787 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq x86_64 0.9.0-1.el8 CentOS-Base 47 k 2020-09-12T13:52:41.787 INFO:teuthology.orchestra.run.smithi027.stdout: fmt x86_64 6.2.1-1.el8 epel 116 k 2020-09-12T13:52:41.787 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs x86_64 2.7-6.el8 epel 315 k 2020-09-12T13:52:41.787 INFO:teuthology.orchestra.run.smithi027.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2020-09-12T13:52:41.787 INFO:teuthology.orchestra.run.smithi027.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2020-09-12T13:52:41.787 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2020-09-12T13:52:41.788 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:52:41.788 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2020-09-12T13:52:41.788 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:52:41.788 INFO:teuthology.orchestra.run.smithi027.stdout:Install 22 Packages 2020-09-12T13:52:41.788 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 2 Packages 2020-09-12T13:52:41.788 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:52:41.789 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 54 M 2020-09-12T13:52:41.789 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2020-09-12T13:52:43.726 INFO:teuthology.orchestra.run.smithi027.stdout:(1/24): ceph-base-16.0.0-5485.gf346cc73d15.el8. 3.1 MB/s | 6.1 MB 00:01 2020-09-12T13:52:43.783 INFO:teuthology.orchestra.run.smithi027.stdout:(2/24): ceph-selinux-16.0.0-5485.gf346cc73d15.e 437 kB/s | 25 kB 00:00 2020-09-12T13:52:43.921 INFO:teuthology.orchestra.run.smithi027.stdout:(3/24): libcephfs2-16.0.0-5485.gf346cc73d15.el8 3.9 MB/s | 553 kB 00:00 2020-09-12T13:52:44.060 INFO:teuthology.orchestra.run.smithi027.stdout:(4/24): libradosstriper1-16.0.0-5485.gf346cc73d 3.4 MB/s | 485 kB 00:00 2020-09-12T13:52:44.985 INFO:teuthology.orchestra.run.smithi027.stdout:(5/24): librgw2-16.0.0-5485.gf346cc73d15.el8.x8 3.9 MB/s | 3.6 MB 00:00 2020-09-12T13:52:45.043 INFO:teuthology.orchestra.run.smithi027.stdout:(6/24): python3-ceph-argparse-16.0.0-5485.gf346 688 kB/s | 40 kB 00:00 2020-09-12T13:52:45.097 INFO:teuthology.orchestra.run.smithi027.stdout:(7/24): python3-ceph-common-16.0.0-5485.gf346cc 1.3 MB/s | 72 kB 00:00 2020-09-12T13:52:45.154 INFO:teuthology.orchestra.run.smithi027.stdout:(8/24): python3-cephfs-16.0.0-5485.gf346cc73d15 3.5 MB/s | 204 kB 00:00 2020-09-12T13:52:45.267 INFO:teuthology.orchestra.run.smithi027.stdout:(9/24): python3-rados-16.0.0-5485.gf346cc73d15. 3.1 MB/s | 359 kB 00:00 2020-09-12T13:52:45.348 INFO:teuthology.orchestra.run.smithi027.stdout:(10/24): python3-rbd-16.0.0-5485.gf346cc73d15.e 4.0 MB/s | 335 kB 00:00 2020-09-12T13:52:45.379 INFO:teuthology.orchestra.run.smithi027.stdout:(11/24): python3-rgw-16.0.0-5485.gf346cc73d15.e 3.5 MB/s | 113 kB 00:00 2020-09-12T13:52:45.415 INFO:teuthology.orchestra.run.smithi027.stdout:(12/24): librdkafka-0.11.4-1.el8.x86_64.rpm 9.6 MB/s | 353 kB 00:00 2020-09-12T13:52:45.420 INFO:teuthology.orchestra.run.smithi027.stdout:(13/24): lttng-ust-2.8.1-11.el8.x86_64.rpm 54 MB/s | 259 kB 00:00 2020-09-12T13:52:45.421 INFO:teuthology.orchestra.run.smithi027.stdout:(14/24): python3-prettytable-0.7.2-14.el8.noarc 34 MB/s | 44 kB 00:00 2020-09-12T13:52:45.423 INFO:teuthology.orchestra.run.smithi027.stdout:(15/24): librabbitmq-0.9.0-1.el8.x86_64.rpm 33 MB/s | 47 kB 00:00 2020-09-12T13:52:45.521 INFO:teuthology.orchestra.run.smithi027.stdout:(16/24): fmt-6.2.1-1.el8.x86_64.rpm 1.2 MB/s | 116 kB 00:00 2020-09-12T13:52:45.669 INFO:teuthology.orchestra.run.smithi027.stdout:(17/24): ceph-common-16.0.0-5485.gf346cc73d15.e 5.9 MB/s | 23 MB 00:03 2020-09-12T13:52:45.672 INFO:teuthology.orchestra.run.smithi027.stdout:(18/24): gperftools-libs-2.7-6.el8.x86_64.rpm 2.0 MB/s | 315 kB 00:00 2020-09-12T13:52:45.692 INFO:teuthology.orchestra.run.smithi027.stdout:(19/24): liboath-2.6.2-3.el8.x86_64.rpm 2.9 MB/s | 59 kB 00:00 2020-09-12T13:52:45.712 INFO:teuthology.orchestra.run.smithi027.stdout:(20/24): libunwind-1.3.1-3.el8.x86_64.rpm 3.6 MB/s | 75 kB 00:00 2020-09-12T13:52:45.750 INFO:teuthology.orchestra.run.smithi027.stdout:(21/24): leveldb-1.22-1.el8.x86_64.rpm 2.2 MB/s | 181 kB 00:00 2020-09-12T13:52:45.901 INFO:teuthology.orchestra.run.smithi027.stdout:(22/24): librbd1-16.0.0-5485.gf346cc73d15.el8.x 14 MB/s | 2.2 MB 00:00 2020-09-12T13:52:46.035 INFO:teuthology.orchestra.run.smithi027.stdout:(23/24): ceph-radosgw-16.0.0-5485.gf346cc73d15. 2.7 MB/s | 12 MB 00:04 2020-09-12T13:52:46.137 INFO:teuthology.orchestra.run.smithi027.stdout:(24/24): librados2-16.0.0-5485.gf346cc73d15.el8 8.8 MB/s | 3.7 MB 00:00 2020-09-12T13:52:46.139 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2020-09-12T13:52:46.140 INFO:teuthology.orchestra.run.smithi027.stdout:Total 12 MB/s | 54 MB 00:04 2020-09-12T13:52:46.145 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2020-09-12T13:52:46.205 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2020-09-12T13:52:46.205 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2020-09-12T13:52:46.907 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2020-09-12T13:52:46.908 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2020-09-12T13:52:47.985 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2020-09-12T13:52:48.140 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fmt-6.2.1-1.el8.x86_64 1/26 2020-09-12T13:52:48.386 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/26 2020-09-12T13:52:48.525 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/26 2020-09-12T13:52:48.964 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/26 2020-09-12T13:52:49.150 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librados2-2:16.0.0-5485.gf346cc73d15.el8.x86_64 4/26 2020-09-12T13:52:49.248 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:16.0.0-5485.gf346cc73d15.el8.x86_64 4/26 2020-09-12T13:52:49.474 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rados-2:16.0.0-5485.gf346cc73d15.el8.x86_6 5/26 2020-09-12T13:52:49.600 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs2-2:16.0.0-5485.gf346cc73d15.el8.x86_64 6/26 2020-09-12T13:52:49.898 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs2-2:16.0.0-5485.gf346cc73d15.el8.x86_64 6/26 2020-09-12T13:52:50.125 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librbd1-2:16.0.0-5485.gf346cc73d15.el8.x86_64 7/26 2020-09-12T13:52:50.196 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:16.0.0-5485.gf346cc73d15.el8.x86_64 7/26 2020-09-12T13:52:50.394 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librabbitmq-0.9.0-1.el8.x86_64 8/26 2020-09-12T13:52:50.558 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 9/26 2020-09-12T13:52:50.954 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 9/26 2020-09-12T13:52:51.117 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librgw2-2:16.0.0-5485.gf346cc73d15.el8.x86_64 10/26 2020-09-12T13:52:51.196 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librgw2-2:16.0.0-5485.gf346cc73d15.el8.x86_64 10/26 2020-09-12T13:52:51.347 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : leveldb-1.22-1.el8.x86_64 11/26 2020-09-12T13:52:51.497 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-argparse-2:16.0.0-5485.gf346cc73d15.e 12/26 2020-09-12T13:52:51.704 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cephfs-2:16.0.0-5485.gf346cc73d15.el8.x86_ 13/26 2020-09-12T13:52:51.875 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rgw-2:16.0.0-5485.gf346cc73d15.el8.x86_64 14/26 2020-09-12T13:52:52.096 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rbd-2:16.0.0-5485.gf346cc73d15.el8.x86_64 15/26 2020-09-12T13:52:52.242 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libradosstriper1-2:16.0.0-5485.gf346cc73d15.el8.x8 16/26 2020-09-12T13:52:52.315 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libradosstriper1-2:16.0.0-5485.gf346cc73d15.el8.x8 16/26 2020-09-12T13:52:52.489 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/26 2020-09-12T13:52:52.675 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : gperftools-libs-2.7-6.el8.x86_64 18/26 2020-09-12T13:52:52.849 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 19/26 2020-09-12T13:52:52.983 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-common-2:16.0.0-5485.gf346cc73d15.el8 20/26 2020-09-12T13:52:56.999 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:16.0.0-5485.gf346cc73d15.el8.x86_64 21/26 2020-09-12T13:52:57.208 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-common-2:16.0.0-5485.gf346cc73d15.el8.x86_64 21/26 2020-09-12T13:52:57.967 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:16.0.0-5485.gf346cc73d15.el8.x86_64 21/26 2020-09-12T13:52:58.259 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-base-2:16.0.0-5485.gf346cc73d15.el8.x86_64 22/26 2020-09-12T13:52:58.351 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-base-2:16.0.0-5485.gf346cc73d15.el8.x86_64 22/26 2020-09-12T13:52:58.499 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-selinux-2:16.0.0-5485.gf346cc73d15.el8.x86_64 23/26 2020-09-12T13:53:15.897 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-selinux-2:16.0.0-5485.gf346cc73d15.el8.x86_64 23/26 2020-09-12T13:53:15.898 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /sys 2020-09-12T13:53:15.898 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /proc 2020-09-12T13:53:15.898 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /dev 2020-09-12T13:53:15.898 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /run 2020-09-12T13:53:15.898 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /mnt 2020-09-12T13:53:15.899 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /var/tmp 2020-09-12T13:53:15.899 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /home 2020-09-12T13:53:15.899 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /tmp 2020-09-12T13:53:15.899 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /dev 2020-09-12T13:53:15.899 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:17.180 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-radosgw-2:16.0.0-5485.gf346cc73d15.el8.x86_64 24/26 2020-09-12T13:53:17.256 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-radosgw-2:16.0.0-5485.gf346cc73d15.el8.x86_64 24/26 2020-09-12T13:53:17.256 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 25/26 2020-09-12T13:53:17.424 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 25/26 2020-09-12T13:53:17.425 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 26/26 2020-09-12T13:53:18.181 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 26/26 2020-09-12T13:53:18.181 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-base-2:16.0.0-5485.gf346cc73d15.el8.x86_64 1/26 2020-09-12T13:53:18.181 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-common-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2/26 2020-09-12T13:53:18.181 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-radosgw-2:16.0.0-5485.gf346cc73d15.el8.x86_64 3/26 2020-09-12T13:53:18.182 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-selinux-2:16.0.0-5485.gf346cc73d15.el8.x86_64 4/26 2020-09-12T13:53:18.182 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs2-2:16.0.0-5485.gf346cc73d15.el8.x86_64 5/26 2020-09-12T13:53:18.182 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libradosstriper1-2:16.0.0-5485.gf346cc73d15.el8.x8 6/26 2020-09-12T13:53:18.182 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librgw2-2:16.0.0-5485.gf346cc73d15.el8.x86_64 7/26 2020-09-12T13:53:18.182 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-argparse-2:16.0.0-5485.gf346cc73d15.e 8/26 2020-09-12T13:53:18.182 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-common-2:16.0.0-5485.gf346cc73d15.el8 9/26 2020-09-12T13:53:18.182 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cephfs-2:16.0.0-5485.gf346cc73d15.el8.x86_ 10/26 2020-09-12T13:53:18.183 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rados-2:16.0.0-5485.gf346cc73d15.el8.x86_6 11/26 2020-09-12T13:53:18.183 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rbd-2:16.0.0-5485.gf346cc73d15.el8.x86_64 12/26 2020-09-12T13:53:18.183 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rgw-2:16.0.0-5485.gf346cc73d15.el8.x86_64 13/26 2020-09-12T13:53:18.183 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 14/26 2020-09-12T13:53:18.183 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/26 2020-09-12T13:53:18.183 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/26 2020-09-12T13:53:18.183 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librabbitmq-0.9.0-1.el8.x86_64 17/26 2020-09-12T13:53:18.184 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 18/26 2020-09-12T13:53:18.184 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : gperftools-libs-2.7-6.el8.x86_64 19/26 2020-09-12T13:53:18.184 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : leveldb-1.22-1.el8.x86_64 20/26 2020-09-12T13:53:18.184 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 21/26 2020-09-12T13:53:18.184 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 22/26 2020-09-12T13:53:18.184 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:16.0.0-5485.gf346cc73d15.el8.x86_64 23/26 2020-09-12T13:53:18.184 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 24/26 2020-09-12T13:53:18.184 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:16.0.0-5485.gf346cc73d15.el8.x86_64 25/26 2020-09-12T13:53:18.872 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 26/26 2020-09-12T13:53:18.873 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:18.873 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2020-09-12T13:53:18.873 INFO:teuthology.orchestra.run.smithi027.stdout: librados2-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:18.873 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:18.873 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:18.873 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2020-09-12T13:53:18.873 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:18.874 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:18.874 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:18.874 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:18.874 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:18.874 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:18.874 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:18.874 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:18.875 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:18.875 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:18.875 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:18.875 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:18.875 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:18.875 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka-0.11.4-1.el8.x86_64 2020-09-12T13:53:18.875 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust-2.8.1-11.el8.x86_64 2020-09-12T13:53:18.876 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable-0.7.2-14.el8.noarch 2020-09-12T13:53:18.876 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq-0.9.0-1.el8.x86_64 2020-09-12T13:53:18.876 INFO:teuthology.orchestra.run.smithi027.stdout: fmt-6.2.1-1.el8.x86_64 2020-09-12T13:53:18.876 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs-2.7-6.el8.x86_64 2020-09-12T13:53:18.876 INFO:teuthology.orchestra.run.smithi027.stdout: leveldb-1.22-1.el8.x86_64 2020-09-12T13:53:18.876 INFO:teuthology.orchestra.run.smithi027.stdout: liboath-2.6.2-3.el8.x86_64 2020-09-12T13:53:18.876 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind-1.3.1-3.el8.x86_64 2020-09-12T13:53:18.876 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:18.877 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:53:19.054 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-test 2020-09-12T13:53:19.505 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:39 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:53:20.225 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:53:20.227 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:53:20.228 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2020-09-12T13:53:20.228 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:53:20.228 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2020-09-12T13:53:20.228 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 47 M 2020-09-12T13:53:20.228 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2020-09-12T13:53:20.228 INFO:teuthology.orchestra.run.smithi027.stdout: jq x86_64 1.5-12.el8 CentOS-AppStream 161 k 2020-09-12T13:53:20.229 INFO:teuthology.orchestra.run.smithi027.stdout: oniguruma x86_64 6.8.2-1.el8 CentOS-AppStream 188 k 2020-09-12T13:53:20.229 INFO:teuthology.orchestra.run.smithi027.stdout: socat x86_64 1.7.3.2-6.el8 CentOS-AppStream 298 k 2020-09-12T13:53:20.229 INFO:teuthology.orchestra.run.smithi027.stdout: libxslt x86_64 1.1.32-3.el8 CentOS-Base 249 k 2020-09-12T13:53:20.229 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet x86_64 1.6.1-11.el8 epel 69 k 2020-09-12T13:53:20.229 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:20.229 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2020-09-12T13:53:20.229 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:53:20.230 INFO:teuthology.orchestra.run.smithi027.stdout:Install 6 Packages 2020-09-12T13:53:20.230 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:20.230 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 48 M 2020-09-12T13:53:20.230 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 204 M 2020-09-12T13:53:20.230 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2020-09-12T13:53:20.264 INFO:teuthology.orchestra.run.smithi027.stdout:(1/6): jq-1.5-12.el8.x86_64.rpm 4.5 MB/s | 161 kB 00:00 2020-09-12T13:53:20.267 INFO:teuthology.orchestra.run.smithi027.stdout:(2/6): oniguruma-6.8.2-1.el8.x86_64.rpm 4.9 MB/s | 188 kB 00:00 2020-09-12T13:53:20.270 INFO:teuthology.orchestra.run.smithi027.stdout:(3/6): socat-1.7.3.2-6.el8.x86_64.rpm 49 MB/s | 298 kB 00:00 2020-09-12T13:53:20.273 INFO:teuthology.orchestra.run.smithi027.stdout:(4/6): libxslt-1.1.32-3.el8.x86_64.rpm 52 MB/s | 249 kB 00:00 2020-09-12T13:53:20.349 INFO:teuthology.orchestra.run.smithi027.stdout:(5/6): xmlstarlet-1.6.1-11.el8.x86_64.rpm 879 kB/s | 69 kB 00:00 2020-09-12T13:53:22.461 INFO:teuthology.orchestra.run.smithi027.stdout:(6/6): ceph-test-16.0.0-5485.gf346cc73d15.el8.x 21 MB/s | 47 MB 00:02 2020-09-12T13:53:22.464 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2020-09-12T13:53:22.465 INFO:teuthology.orchestra.run.smithi027.stdout:Total 22 MB/s | 48 MB 00:02 2020-09-12T13:53:22.472 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2020-09-12T13:53:22.499 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2020-09-12T13:53:22.499 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2020-09-12T13:53:22.851 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2020-09-12T13:53:22.852 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2020-09-12T13:53:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2020-09-12T13:53:23.545 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libxslt-1.1.32-3.el8.x86_64 1/6 2020-09-12T13:53:23.717 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xmlstarlet-1.6.1-11.el8.x86_64 2/6 2020-09-12T13:53:23.892 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : socat-1.7.3.2-6.el8.x86_64 3/6 2020-09-12T13:53:24.063 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : oniguruma-6.8.2-1.el8.x86_64 4/6 2020-09-12T13:53:24.139 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: oniguruma-6.8.2-1.el8.x86_64 4/6 2020-09-12T13:53:28.848 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : jq-1.5-12.el8.x86_64 5/6 2020-09-12T13:53:29.093 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-test-2:16.0.0-5485.gf346cc73d15.el8.x86_64 6/6 2020-09-12T13:53:29.285 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-test-2:16.0.0-5485.gf346cc73d15.el8.x86_64 6/6 2020-09-12T13:53:29.286 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-test-2:16.0.0-5485.gf346cc73d15.el8.x86_64 1/6 2020-09-12T13:53:29.286 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : jq-1.5-12.el8.x86_64 2/6 2020-09-12T13:53:29.286 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : oniguruma-6.8.2-1.el8.x86_64 3/6 2020-09-12T13:53:29.286 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : socat-1.7.3.2-6.el8.x86_64 4/6 2020-09-12T13:53:29.286 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libxslt-1.1.32-3.el8.x86_64 5/6 2020-09-12T13:53:29.587 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 6/6 2020-09-12T13:53:29.587 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:29.587 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2020-09-12T13:53:29.587 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:29.588 INFO:teuthology.orchestra.run.smithi027.stdout: jq-1.5-12.el8.x86_64 2020-09-12T13:53:29.588 INFO:teuthology.orchestra.run.smithi027.stdout: oniguruma-6.8.2-1.el8.x86_64 2020-09-12T13:53:29.588 INFO:teuthology.orchestra.run.smithi027.stdout: socat-1.7.3.2-6.el8.x86_64 2020-09-12T13:53:29.588 INFO:teuthology.orchestra.run.smithi027.stdout: libxslt-1.1.32-3.el8.x86_64 2020-09-12T13:53:29.588 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet-1.6.1-11.el8.x86_64 2020-09-12T13:53:29.588 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:29.589 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:53:29.845 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph 2020-09-12T13:53:30.284 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:50 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:53:30.689 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:53:30.693 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:53:30.693 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2020-09-12T13:53:30.694 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:53:30.694 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2020-09-12T13:53:30.694 INFO:teuthology.orchestra.run.smithi027.stdout: ceph x86_64 2:16.0.0-5485.gf346cc73d15.el8 2020-09-12T13:53:30.694 INFO:teuthology.orchestra.run.smithi027.stdout: ceph 6.5 k 2020-09-12T13:53:30.694 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2020-09-12T13:53:30.694 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds x86_64 2:16.0.0-5485.gf346cc73d15.el8 2020-09-12T13:53:30.695 INFO:teuthology.orchestra.run.smithi027.stdout: ceph 2.2 M 2020-09-12T13:53:30.695 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr x86_64 2:16.0.0-5485.gf346cc73d15.el8 2020-09-12T13:53:30.695 INFO:teuthology.orchestra.run.smithi027.stdout: ceph 1.5 M 2020-09-12T13:53:30.695 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon x86_64 2:16.0.0-5485.gf346cc73d15.el8 2020-09-12T13:53:30.695 INFO:teuthology.orchestra.run.smithi027.stdout: ceph 4.9 M 2020-09-12T13:53:30.695 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd x86_64 2:16.0.0-5485.gf346cc73d15.el8 2020-09-12T13:53:30.695 INFO:teuthology.orchestra.run.smithi027.stdout: ceph 18 M 2020-09-12T13:53:30.696 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core noarch 2:16.0.0-5485.gf346cc73d15.el8 2020-09-12T13:53:30.696 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-noarch 189 k 2020-09-12T13:53:30.696 INFO:teuthology.orchestra.run.smithi027.stdout: python3-lxml x86_64 4.2.3-1.el8 CentOS-AppStream 1.5 M 2020-09-12T13:53:30.696 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2020-09-12T13:53:30.696 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2020-09-12T13:53:30.696 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2020-09-12T13:53:30.696 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webencodings noarch 0.5.1-6.el8 CentOS-AppStream 27 k 2020-09-12T13:53:30.696 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2020-09-12T13:53:30.697 INFO:teuthology.orchestra.run.smithi027.stdout: python3-chardet noarch 3.0.4-7.el8 CentOS-Base 195 k 2020-09-12T13:53:30.697 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pysocks noarch 1.6.8-3.el8 CentOS-Base 34 k 2020-09-12T13:53:30.697 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests noarch 2.20.0-2.1.el8_1 CentOS-Base 123 k 2020-09-12T13:53:30.697 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3 noarch 1.24.2-2.el8 CentOS-Base 176 k 2020-09-12T13:53:30.697 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2020-09-12T13:53:30.697 INFO:teuthology.orchestra.run.smithi027.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2020-09-12T13:53:30.697 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot noarch 8.2.1-1.el8 epel 145 k 2020-09-12T13:53:30.698 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2020-09-12T13:53:30.698 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2020-09-12T13:53:30.698 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2020-09-12T13:53:30.698 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2020-09-12T13:53:30.698 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2020-09-12T13:53:30.698 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2020-09-12T13:53:30.698 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2020-09-12T13:53:30.699 INFO:teuthology.orchestra.run.smithi027.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2020-09-12T13:53:30.699 INFO:teuthology.orchestra.run.smithi027.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2020-09-12T13:53:30.699 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2020-09-12T13:53:30.699 INFO:teuthology.orchestra.run.smithi027.stdout: python3-trustme noarch 0.5.2-4.el8 epel 26 k 2020-09-12T13:53:30.699 INFO:teuthology.orchestra.run.smithi027.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2020-09-12T13:53:30.699 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2020-09-12T13:53:30.699 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2020-09-12T13:53:30.700 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2020-09-12T13:53:30.700 INFO:teuthology.orchestra.run.smithi027.stdout:Installing weak dependencies: 2020-09-12T13:53:30.700 INFO:teuthology.orchestra.run.smithi027.stdout: python3-html5lib noarch 1:0.999999999-6.el8 CentOS-AppStream 214 k 2020-09-12T13:53:30.700 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cssselect noarch 0.9.2-10.el8 epel 40 k 2020-09-12T13:53:30.700 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:30.700 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2020-09-12T13:53:30.700 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:53:30.701 INFO:teuthology.orchestra.run.smithi027.stdout:Install 36 Packages 2020-09-12T13:53:30.701 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:30.701 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 31 M 2020-09-12T13:53:30.701 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 110 M 2020-09-12T13:53:30.702 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2020-09-12T13:53:30.869 INFO:teuthology.orchestra.run.smithi027.stdout:(1/36): ceph-16.0.0-5485.gf346cc73d15.el8.x86_6 38 kB/s | 6.5 kB 00:00 2020-09-12T13:53:31.124 INFO:teuthology.orchestra.run.smithi027.stdout:(2/36): ceph-mds-16.0.0-5485.gf346cc73d15.el8.x 5.1 MB/s | 2.2 MB 00:00 2020-09-12T13:53:31.133 INFO:teuthology.orchestra.run.smithi027.stdout:(3/36): ceph-mgr-16.0.0-5485.gf346cc73d15.el8.x 3.6 MB/s | 1.5 MB 00:00 2020-09-12T13:53:31.198 INFO:teuthology.orchestra.run.smithi027.stdout:(4/36): ceph-mgr-modules-core-16.0.0-5485.gf346 2.8 MB/s | 189 kB 00:00 2020-09-12T13:53:31.299 INFO:teuthology.orchestra.run.smithi027.stdout:(5/36): python3-html5lib-0.999999999-6.el8.noar 2.1 MB/s | 214 kB 00:00 2020-09-12T13:53:31.325 INFO:teuthology.orchestra.run.smithi027.stdout:(6/36): python3-lxml-4.2.3-1.el8.x86_64.rpm 59 MB/s | 1.5 MB 00:00 2020-09-12T13:53:31.414 INFO:teuthology.orchestra.run.smithi027.stdout:(7/36): python3-mako-1.0.6-13.el8.noarch.rpm 1.7 MB/s | 157 kB 00:00 2020-09-12T13:53:31.459 INFO:teuthology.orchestra.run.smithi027.stdout:(8/36): ceph-mon-16.0.0-5485.gf346cc73d15.el8.x 8.3 MB/s | 4.9 MB 00:00 2020-09-12T13:53:31.473 INFO:teuthology.orchestra.run.smithi027.stdout:(9/36): python3-markupsafe-0.23-19.el8.x86_64.r 668 kB/s | 39 kB 00:00 2020-09-12T13:53:31.533 INFO:teuthology.orchestra.run.smithi027.stdout:(10/36): python3-pytz-2017.2-9.el8.noarch.rpm 729 kB/s | 54 kB 00:00 2020-09-12T13:53:31.552 INFO:teuthology.orchestra.run.smithi027.stdout:(11/36): python3-werkzeug-0.12.2-4.el8.noarch.r 31 MB/s | 457 kB 00:00 2020-09-12T13:53:31.583 INFO:teuthology.orchestra.run.smithi027.stdout:(12/36): python3-chardet-3.0.4-7.el8.noarch.rpm 6.3 MB/s | 195 kB 00:00 2020-09-12T13:53:31.605 INFO:teuthology.orchestra.run.smithi027.stdout:(13/36): python3-webencodings-0.5.1-6.el8.noarc 203 kB/s | 27 kB 00:00 2020-09-12T13:53:31.637 INFO:teuthology.orchestra.run.smithi027.stdout:(14/36): python3-pysocks-1.6.8-3.el8.noarch.rpm 640 kB/s | 34 kB 00:00 2020-09-12T13:53:31.647 INFO:teuthology.orchestra.run.smithi027.stdout:(15/36): python3-requests-2.20.0-2.1.el8_1.noar 2.9 MB/s | 123 kB 00:00 2020-09-12T13:53:31.653 INFO:teuthology.orchestra.run.smithi027.stdout:(16/36): python3-urllib3-1.24.2-2.el8.noarch.rp 11 MB/s | 176 kB 00:00 2020-09-12T13:53:31.725 INFO:teuthology.orchestra.run.smithi027.stdout:(17/36): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 559 kB/s | 44 kB 00:00 2020-09-12T13:53:31.733 INFO:teuthology.orchestra.run.smithi027.stdout:(18/36): python3-beautifulsoup4-4.6.3-2.el8.1.n 2.2 MB/s | 185 kB 00:00 2020-09-12T13:53:31.765 INFO:teuthology.orchestra.run.smithi027.stdout:(19/36): python3-cheroot-8.2.1-1.el8.noarch.rpm 3.6 MB/s | 145 kB 00:00 2020-09-12T13:53:31.768 INFO:teuthology.orchestra.run.smithi027.stdout:(20/36): python3-cherrypy-18.4.0-1.el8.noarch.r 11 MB/s | 384 kB 00:00 2020-09-12T13:53:31.784 INFO:teuthology.orchestra.run.smithi027.stdout:(21/36): python3-jaraco-6.2-6.el8.noarch.rpm 686 kB/s | 11 kB 00:00 2020-09-12T13:53:31.785 INFO:teuthology.orchestra.run.smithi027.stdout:(22/36): python3-cssselect-0.9.2-10.el8.noarch. 2.0 MB/s | 40 kB 00:00 2020-09-12T13:53:31.800 INFO:teuthology.orchestra.run.smithi027.stdout:(23/36): python3-jaraco-functools-2.0-4.el8.noa 1.0 MB/s | 18 kB 00:00 2020-09-12T13:53:31.805 INFO:teuthology.orchestra.run.smithi027.stdout:(24/36): python3-logutils-0.3.5-11.el8.noarch.r 2.4 MB/s | 49 kB 00:00 2020-09-12T13:53:31.817 INFO:teuthology.orchestra.run.smithi027.stdout:(25/36): python3-more-itertools-7.2.0-3.el8.noa 3.4 MB/s | 59 kB 00:00 2020-09-12T13:53:31.834 INFO:teuthology.orchestra.run.smithi027.stdout:(26/36): python3-portend-2.6-1.el8.noarch.rpm 962 kB/s | 16 kB 00:00 2020-09-12T13:53:31.845 INFO:teuthology.orchestra.run.smithi027.stdout:(27/36): python3-pecan-1.3.2-9.el8.noarch.rpm 7.0 MB/s | 283 kB 00:00 2020-09-12T13:53:31.850 INFO:teuthology.orchestra.run.smithi027.stdout:(28/36): python3-simplegeneric-0.8.1-17.el8.noa 1.1 MB/s | 19 kB 00:00 2020-09-12T13:53:31.864 INFO:teuthology.orchestra.run.smithi027.stdout:(29/36): python3-singledispatch-3.4.0.3-18.el8. 1.2 MB/s | 24 kB 00:00 2020-09-12T13:53:31.866 INFO:teuthology.orchestra.run.smithi027.stdout:(30/36): python3-tempora-1.14.1-5.el8.noarch.rp 1.8 MB/s | 29 kB 00:00 2020-09-12T13:53:31.885 INFO:teuthology.orchestra.run.smithi027.stdout:(31/36): python3-waitress-1.2.1-2.el8.1.noarch. 13 MB/s | 241 kB 00:00 2020-09-12T13:53:31.885 INFO:teuthology.orchestra.run.smithi027.stdout:(32/36): python3-trustme-0.5.2-4.el8.noarch.rpm 1.3 MB/s | 26 kB 00:00 2020-09-12T13:53:31.903 INFO:teuthology.orchestra.run.smithi027.stdout:(33/36): python3-webob-1.8.5-1.el8.1.noarch.rpm 13 MB/s | 251 kB 00:00 2020-09-12T13:53:31.905 INFO:teuthology.orchestra.run.smithi027.stdout:(34/36): python3-webtest-2.0.33-1.el8.noarch.rp 4.2 MB/s | 85 kB 00:00 2020-09-12T13:53:31.920 INFO:teuthology.orchestra.run.smithi027.stdout:(35/36): python3-zc-lockfile-2.0-2.el8.noarch.r 1.4 MB/s | 23 kB 00:00 2020-09-12T13:53:32.116 INFO:teuthology.orchestra.run.smithi027.stdout:(36/36): ceph-osd-16.0.0-5485.gf346cc73d15.el8. 18 MB/s | 18 MB 00:00 2020-09-12T13:53:32.120 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2020-09-12T13:53:32.120 INFO:teuthology.orchestra.run.smithi027.stdout:Total 22 MB/s | 31 MB 00:01 2020-09-12T13:53:32.132 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2020-09-12T13:53:32.177 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2020-09-12T13:53:32.177 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2020-09-12T13:53:32.534 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2020-09-12T13:53:32.534 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2020-09-12T13:53:33.911 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2020-09-12T13:53:34.121 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/36 2020-09-12T13:53:34.336 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/36 2020-09-12T13:53:34.563 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/36 2020-09-12T13:53:34.778 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/36 2020-09-12T13:53:34.902 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-trustme-0.5.2-4.el8.noarch 5/36 2020-09-12T13:53:35.108 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 6/36 2020-09-12T13:53:35.273 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 7/36 2020-09-12T13:53:35.423 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 8/36 2020-09-12T13:53:35.593 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 9/36 2020-09-12T13:53:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 10/36 2020-09-12T13:53:36.083 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cheroot-8.2.1-1.el8.noarch 11/36 2020-09-12T13:53:36.282 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cssselect-0.9.2-10.el8.noarch 12/36 2020-09-12T13:53:36.469 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 13/36 2020-09-12T13:53:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pysocks-1.6.8-3.el8.noarch 14/36 2020-09-12T13:53:36.945 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-urllib3-1.24.2-2.el8.noarch 15/36 2020-09-12T13:53:37.171 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-chardet-3.0.4-7.el8.noarch 16/36 2020-09-12T13:53:37.439 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-2.20.0-2.1.el8_1.noarch 17/36 2020-09-12T13:53:37.657 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 18/36 2020-09-12T13:53:37.887 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webencodings-0.5.1-6.el8.noarch 19/36 2020-09-12T13:53:38.068 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-html5lib-1:0.999999999-6.el8.noarch 20/36 2020-09-12T13:53:38.439 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 21/36 2020-09-12T13:53:38.722 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-lxml-4.2.3-1.el8.x86_64 22/36 2020-09-12T13:53:38.884 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 23/36 2020-09-12T13:53:39.056 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/36 2020-09-12T13:53:39.216 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 25/36 2020-09-12T13:53:39.432 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-portend-2.6-1.el8.noarch 26/36 2020-09-12T13:53:39.657 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 27/36 2020-09-12T13:53:39.918 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/36 2020-09-12T13:53:40.247 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/36 2020-09-12T13:53:40.697 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 30/36 2020-09-12T13:53:41.058 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-modules-core-2:16.0.0-5485.gf346cc73d15.e 31/36 2020-09-12T13:53:41.203 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-2:16.0.0-5485.gf346cc73d15.el8.x86_64 32/36 2020-09-12T13:53:42.808 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-2:16.0.0-5485.gf346cc73d15.el8.x86_64 32/36 2020-09-12T13:53:43.002 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-osd-2:16.0.0-5485.gf346cc73d15.el8.x86_64 33/36 2020-09-12T13:53:43.463 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-osd-2:16.0.0-5485.gf346cc73d15.el8.x86_64 33/36 2020-09-12T13:53:43.644 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mon-2:16.0.0-5485.gf346cc73d15.el8.x86_64 34/36 2020-09-12T13:53:43.856 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mon-2:16.0.0-5485.gf346cc73d15.el8.x86_64 34/36 2020-09-12T13:53:43.985 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mds-2:16.0.0-5485.gf346cc73d15.el8.x86_64 35/36 2020-09-12T13:53:44.003 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mds-2:16.0.0-5485.gf346cc73d15.el8.x86_64 35/36 2020-09-12T13:53:44.142 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-2:16.0.0-5485.gf346cc73d15.el8.x86_64 36/36 2020-09-12T13:53:44.458 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-2:16.0.0-5485.gf346cc73d15.el8.x86_64 36/36 2020-09-12T13:53:44.458 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-2:16.0.0-5485.gf346cc73d15.el8.x86_64 1/36 2020-09-12T13:53:44.458 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mds-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2/36 2020-09-12T13:53:44.459 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-2:16.0.0-5485.gf346cc73d15.el8.x86_64 3/36 2020-09-12T13:53:44.459 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mon-2:16.0.0-5485.gf346cc73d15.el8.x86_64 4/36 2020-09-12T13:53:44.459 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-osd-2:16.0.0-5485.gf346cc73d15.el8.x86_64 5/36 2020-09-12T13:53:44.459 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-modules-core-2:16.0.0-5485.gf346cc73d15.e 6/36 2020-09-12T13:53:44.459 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 7/36 2020-09-12T13:53:44.459 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-lxml-4.2.3-1.el8.x86_64 8/36 2020-09-12T13:53:44.459 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 9/36 2020-09-12T13:53:44.460 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 10/36 2020-09-12T13:53:44.460 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 11/36 2020-09-12T13:53:44.460 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 12/36 2020-09-12T13:53:44.460 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 13/36 2020-09-12T13:53:44.460 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-chardet-3.0.4-7.el8.noarch 14/36 2020-09-12T13:53:44.460 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pysocks-1.6.8-3.el8.noarch 15/36 2020-09-12T13:53:44.460 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-2.20.0-2.1.el8_1.noarch 16/36 2020-09-12T13:53:44.461 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-urllib3-1.24.2-2.el8.noarch 17/36 2020-09-12T13:53:44.461 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/36 2020-09-12T13:53:44.461 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 19/36 2020-09-12T13:53:44.461 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cheroot-8.2.1-1.el8.noarch 20/36 2020-09-12T13:53:44.461 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 21/36 2020-09-12T13:53:44.461 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 22/36 2020-09-12T13:53:44.461 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 23/36 2020-09-12T13:53:44.462 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 24/36 2020-09-12T13:53:44.462 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 25/36 2020-09-12T13:53:44.462 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 26/36 2020-09-12T13:53:44.462 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 27/36 2020-09-12T13:53:44.463 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-portend-2.6-1.el8.noarch 28/36 2020-09-12T13:53:44.463 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 29/36 2020-09-12T13:53:44.463 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 30/36 2020-09-12T13:53:44.464 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 31/36 2020-09-12T13:53:44.464 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-trustme-0.5.2-4.el8.noarch 32/36 2020-09-12T13:53:44.464 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 33/36 2020-09-12T13:53:44.464 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 34/36 2020-09-12T13:53:44.464 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 35/36 2020-09-12T13:53:45.273 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 36/36 2020-09-12T13:53:45.273 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:45.274 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2020-09-12T13:53:45.274 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:45.274 INFO:teuthology.orchestra.run.smithi027.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2020-09-12T13:53:45.274 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cssselect-0.9.2-10.el8.noarch 2020-09-12T13:53:45.274 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:45.274 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:45.275 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:45.275 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:45.275 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core-2:16.0.0-5485.gf346cc73d15.el8.noarch 2020-09-12T13:53:45.275 INFO:teuthology.orchestra.run.smithi027.stdout: python3-lxml-4.2.3-1.el8.x86_64 2020-09-12T13:53:45.275 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako-1.0.6-13.el8.noarch 2020-09-12T13:53:45.275 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe-0.23-19.el8.x86_64 2020-09-12T13:53:45.275 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz-2017.2-9.el8.noarch 2020-09-12T13:53:45.275 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webencodings-0.5.1-6.el8.noarch 2020-09-12T13:53:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2020-09-12T13:53:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: python3-chardet-3.0.4-7.el8.noarch 2020-09-12T13:53:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pysocks-1.6.8-3.el8.noarch 2020-09-12T13:53:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-2.20.0-2.1.el8_1.noarch 2020-09-12T13:53:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3-1.24.2-2.el8.noarch 2020-09-12T13:53:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2020-09-12T13:53:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2020-09-12T13:53:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot-8.2.1-1.el8.noarch 2020-09-12T13:53:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2020-09-12T13:53:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-6.2-6.el8.noarch 2020-09-12T13:53:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2020-09-12T13:53:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils-0.3.5-11.el8.noarch 2020-09-12T13:53:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2020-09-12T13:53:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan-1.3.2-9.el8.noarch 2020-09-12T13:53:45.278 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend-2.6-1.el8.noarch 2020-09-12T13:53:45.278 INFO:teuthology.orchestra.run.smithi027.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2020-09-12T13:53:45.278 INFO:teuthology.orchestra.run.smithi027.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2020-09-12T13:53:45.278 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora-1.14.1-5.el8.noarch 2020-09-12T13:53:45.278 INFO:teuthology.orchestra.run.smithi027.stdout: python3-trustme-0.5.2-4.el8.noarch 2020-09-12T13:53:45.278 INFO:teuthology.orchestra.run.smithi027.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2020-09-12T13:53:45.278 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob-1.8.5-1.el8.1.noarch 2020-09-12T13:53:45.279 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webtest-2.0.33-1.el8.noarch 2020-09-12T13:53:45.279 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2020-09-12T13:53:45.279 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:45.279 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:53:45.564 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-base 2020-09-12T13:53:46.011 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:06 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:53:46.300 INFO:teuthology.orchestra.run.smithi027.stdout:Package ceph-base-2:16.0.0-5485.gf346cc73d15.el8.x86_64 is already installed. 2020-09-12T13:53:46.418 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:53:46.419 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2020-09-12T13:53:46.419 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:53:46.552 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install cephadm 2020-09-12T13:53:46.989 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:06 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:53:47.670 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:53:47.671 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:53:47.671 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repo Size 2020-09-12T13:53:47.671 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:53:47.672 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2020-09-12T13:53:47.672 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 56 k 2020-09-12T13:53:47.672 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:47.672 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2020-09-12T13:53:47.672 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:53:47.672 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2020-09-12T13:53:47.672 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:47.673 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 56 k 2020-09-12T13:53:47.673 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 208 k 2020-09-12T13:53:47.673 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2020-09-12T13:53:47.819 INFO:teuthology.orchestra.run.smithi027.stdout:cephadm-16.0.0-5485.gf346cc73d15.el8.x86_64.rpm 379 kB/s | 56 kB 00:00 2020-09-12T13:53:47.819 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2020-09-12T13:53:47.819 INFO:teuthology.orchestra.run.smithi027.stdout:Total 376 kB/s | 56 kB 00:00 2020-09-12T13:53:47.821 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2020-09-12T13:53:47.830 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2020-09-12T13:53:47.830 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2020-09-12T13:53:47.832 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2020-09-12T13:53:47.832 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2020-09-12T13:53:47.880 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2020-09-12T13:53:49.431 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:16.0.0-5485.gf346cc73d15.el8.x86_64 1/1 2020-09-12T13:53:49.564 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephadm-2:16.0.0-5485.gf346cc73d15.el8.x86_64 1/1 2020-09-12T13:53:49.714 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:16.0.0-5485.gf346cc73d15.el8.x86_64 1/1 2020-09-12T13:53:49.969 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephadm-2:16.0.0-5485.gf346cc73d15.el8.x86_64 1/1 2020-09-12T13:53:49.969 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:49.970 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2020-09-12T13:53:49.970 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:49.970 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:49.970 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:53:50.265 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-immutable-object-cache 2020-09-12T13:53:50.715 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:10 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:53:51.100 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:53:51.100 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:53:51.101 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repo Size 2020-09-12T13:53:51.101 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:53:51.101 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2020-09-12T13:53:51.101 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 203 k 2020-09-12T13:53:51.102 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:51.102 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2020-09-12T13:53:51.102 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:53:51.102 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2020-09-12T13:53:51.102 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:51.102 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 203 k 2020-09-12T13:53:51.102 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 692 k 2020-09-12T13:53:51.103 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2020-09-12T13:53:51.293 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-immutable-object-cache-16.0.0-5485.gf346cc 1.0 MB/s | 203 kB 00:00 2020-09-12T13:53:51.294 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2020-09-12T13:53:51.294 INFO:teuthology.orchestra.run.smithi027.stdout:Total 1.0 MB/s | 203 kB 00:00 2020-09-12T13:53:51.296 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2020-09-12T13:53:51.310 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2020-09-12T13:53:51.310 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2020-09-12T13:53:51.361 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2020-09-12T13:53:51.361 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2020-09-12T13:53:51.476 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2020-09-12T13:53:51.643 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-immutable-object-cache-2:16.0.0-5485.gf346cc73d 1/1 2020-09-12T13:53:51.977 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.0.0-5485.gf346cc73d 1/1 2020-09-12T13:53:52.236 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-immutable-object-cache-2:16.0.0-5485.gf346cc73d 1/1 2020-09-12T13:53:52.236 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:52.236 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2020-09-12T13:53:52.236 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:53:52.236 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:52.237 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:53:52.474 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr 2020-09-12T13:53:52.923 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:12 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:53:53.450 INFO:teuthology.orchestra.run.smithi027.stdout:Package ceph-mgr-2:16.0.0-5485.gf346cc73d15.el8.x86_64 is already installed. 2020-09-12T13:53:53.569 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:53:53.570 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2020-09-12T13:53:53.570 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:53:53.705 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr-dashboard 2020-09-12T13:53:54.144 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:14 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:53:54.531 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:53:54.532 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:53:54.533 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2020-09-12T13:53:54.533 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:53:54.533 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2020-09-12T13:53:54.533 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard noarch 2:16.0.0-5485.gf346cc73d15.el8 ceph-noarch 3.2 M 2020-09-12T13:53:54.533 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2020-09-12T13:53:54.533 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards 2020-09-12T13:53:54.533 INFO:teuthology.orchestra.run.smithi027.stdout: noarch 2:16.0.0-5485.gf346cc73d15.el8 ceph-noarch 24 k 2020-09-12T13:53:54.534 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts noarch 2:16.0.0-5485.gf346cc73d15.el8 ceph-noarch 8.8 k 2020-09-12T13:53:54.534 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-Base 43 k 2020-09-12T13:53:54.534 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2020-09-12T13:53:54.534 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2020-09-12T13:53:54.534 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:54.534 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2020-09-12T13:53:54.534 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:53:54.535 INFO:teuthology.orchestra.run.smithi027.stdout:Install 6 Packages 2020-09-12T13:53:54.535 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:54.535 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 3.5 M 2020-09-12T13:53:54.535 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 14 M 2020-09-12T13:53:54.536 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2020-09-12T13:53:54.645 INFO:teuthology.orchestra.run.smithi027.stdout:(1/6): ceph-prometheus-alerts-16.0.0-5485.gf346 80 kB/s | 8.8 kB 00:00 2020-09-12T13:53:54.659 INFO:teuthology.orchestra.run.smithi027.stdout:(2/6): ceph-grafana-dashboards-16.0.0-5485.gf34 193 kB/s | 24 kB 00:00 2020-09-12T13:53:54.677 INFO:teuthology.orchestra.run.smithi027.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 1.3 MB/s | 43 kB 00:00 2020-09-12T13:53:54.719 INFO:teuthology.orchestra.run.smithi027.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 560 kB/s | 34 kB 00:00 2020-09-12T13:53:54.758 INFO:teuthology.orchestra.run.smithi027.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 2.4 MB/s | 196 kB 00:00 2020-09-12T13:53:54.858 INFO:teuthology.orchestra.run.smithi027.stdout:(6/6): ceph-mgr-dashboard-16.0.0-5485.gf346cc73 9.9 MB/s | 3.2 MB 00:00 2020-09-12T13:53:54.859 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2020-09-12T13:53:54.860 INFO:teuthology.orchestra.run.smithi027.stdout:Total 11 MB/s | 3.5 MB 00:00 2020-09-12T13:53:54.863 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2020-09-12T13:53:54.873 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2020-09-12T13:53:54.873 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2020-09-12T13:53:55.034 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2020-09-12T13:53:55.035 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2020-09-12T13:53:55.306 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2020-09-12T13:53:55.534 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2020-09-12T13:53:55.745 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2020-09-12T13:53:55.941 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2020-09-12T13:53:56.074 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-prometheus-alerts-2:16.0.0-5485.gf346cc73d15.el 4/6 2020-09-12T13:53:57.125 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-grafana-dashboards-2:16.0.0-5485.gf346cc73d15.e 5/6 2020-09-12T13:53:57.409 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-dashboard-2:16.0.0-5485.gf346cc73d15.el8.no 6/6 2020-09-12T13:53:57.604 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.0.0-5485.gf346cc73d15.el8.no 6/6 2020-09-12T13:53:57.605 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-grafana-dashboards-2:16.0.0-5485.gf346cc73d15.e 1/6 2020-09-12T13:53:57.605 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-dashboard-2:16.0.0-5485.gf346cc73d15.el8.no 2/6 2020-09-12T13:53:57.605 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-prometheus-alerts-2:16.0.0-5485.gf346cc73d15.el 3/6 2020-09-12T13:53:57.605 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2020-09-12T13:53:57.605 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2020-09-12T13:53:57.862 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2020-09-12T13:53:57.862 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:57.863 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2020-09-12T13:53:57.863 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard-2:16.0.0-5485.gf346cc73d15.el8.noarch 2020-09-12T13:53:57.863 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards-2:16.0.0-5485.gf346cc73d15.el8.noarch 2020-09-12T13:53:57.863 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts-2:16.0.0-5485.gf346cc73d15.el8.noarch 2020-09-12T13:53:57.863 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jwt-1.6.1-2.el8.noarch 2020-09-12T13:53:57.863 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru-0.7-6.el8.noarch 2020-09-12T13:53:57.863 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes-2.4.1-12.el8.noarch 2020-09-12T13:53:57.864 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:57.864 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:53:57.998 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr-diskprediction-local 2020-09-12T13:53:58.439 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:18 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:53:59.566 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:53:59.567 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:53:59.567 INFO:teuthology.orchestra.run.smithi027.stdout: Package ArchVersion Repository Size 2020-09-12T13:53:59.567 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:53:59.567 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2020-09-12T13:53:59.568 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local noarch2:16.0.0-5485.gf346cc73d15.el8 ceph-noarch 8.6 M 2020-09-12T13:53:59.568 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2020-09-12T13:53:59.568 INFO:teuthology.orchestra.run.smithi027.stdout: openblas x86_640.3.3-2.el8 CentOS-AppStream 4.3 M 2020-09-12T13:53:59.568 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-threads x86_640.3.3-2.el8 CentOS-AppStream 4.4 M 2020-09-12T13:53:59.568 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy x86_641:1.14.3-9.el8 CentOS-AppStream 3.7 M 2020-09-12T13:53:59.568 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py x86_641:1.14.3-9.el8 CentOS-AppStream 225 k 2020-09-12T13:53:59.568 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy x86_641.0.0-20.module_el8.1.0+245+c39af44f CentOS-AppStream 15 M 2020-09-12T13:53:59.568 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran x86_648.3.1-4.5.el8 CentOS-Base 638 k 2020-09-12T13:53:59.569 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath x86_648.3.1-4.5.el8 CentOS-Base 169 k 2020-09-12T13:53:59.569 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:59.569 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2020-09-12T13:53:59.569 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:53:59.569 INFO:teuthology.orchestra.run.smithi027.stdout:Install 8 Packages 2020-09-12T13:53:59.569 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:53:59.570 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 37 M 2020-09-12T13:53:59.570 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 205 M 2020-09-12T13:53:59.570 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2020-09-12T13:53:59.717 INFO:teuthology.orchestra.run.smithi027.stdout:(1/8): openblas-0.3.3-2.el8.x86_64.rpm 29 MB/s | 4.3 MB 00:00 2020-09-12T13:53:59.771 INFO:teuthology.orchestra.run.smithi027.stdout:(2/8): openblas-threads-0.3.3-2.el8.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2020-09-12T13:53:59.820 INFO:teuthology.orchestra.run.smithi027.stdout:(3/8): python3-numpy-f2py-1.14.3-9.el8.x86_64.r 4.5 MB/s | 225 kB 00:00 2020-09-12T13:53:59.891 INFO:teuthology.orchestra.run.smithi027.stdout:(4/8): python3-numpy-1.14.3-9.el8.x86_64.rpm 21 MB/s | 3.7 MB 00:00 2020-09-12T13:53:59.925 INFO:teuthology.orchestra.run.smithi027.stdout:(5/8): libgfortran-8.3.1-4.5.el8.x86_64.rpm 19 MB/s | 638 kB 00:00 2020-09-12T13:53:59.983 INFO:teuthology.orchestra.run.smithi027.stdout:(6/8): libquadmath-8.3.1-4.5.el8.x86_64.rpm 2.8 MB/s | 169 kB 00:00 2020-09-12T13:54:00.202 INFO:teuthology.orchestra.run.smithi027.stdout:(7/8): python3-scipy-1.0.0-20.module_el8.1.0+24 38 MB/s | 15 MB 00:00 2020-09-12T13:54:00.229 INFO:teuthology.orchestra.run.smithi027.stdout:(8/8): ceph-mgr-diskprediction-local-16.0.0-548 13 MB/s | 8.6 MB 00:00 2020-09-12T13:54:00.231 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2020-09-12T13:54:00.231 INFO:teuthology.orchestra.run.smithi027.stdout:Total 55 MB/s | 37 MB 00:00 2020-09-12T13:54:00.243 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2020-09-12T13:54:00.262 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2020-09-12T13:54:00.263 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2020-09-12T13:54:00.722 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2020-09-12T13:54:00.722 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2020-09-12T13:54:01.303 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2020-09-12T13:54:01.466 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libquadmath-8.3.1-4.5.el8.x86_64 1/8 2020-09-12T13:54:01.615 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libquadmath-8.3.1-4.5.el8.x86_64 1/8 2020-09-12T13:54:01.883 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libgfortran-8.3.1-4.5.el8.x86_64 2/8 2020-09-12T13:54:02.461 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libgfortran-8.3.1-4.5.el8.x86_64 2/8 2020-09-12T13:54:02.649 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-0.3.3-2.el8.x86_64 3/8 2020-09-12T13:54:03.226 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: openblas-0.3.3-2.el8.x86_64 3/8 2020-09-12T13:54:03.391 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-threads-0.3.3-2.el8.x86_64 4/8 2020-09-12T13:54:04.143 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: openblas-threads-0.3.3-2.el8.x86_64 4/8 2020-09-12T13:54:04.492 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 5/8 2020-09-12T13:54:04.699 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2020-09-12T13:54:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2020-09-12T13:54:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af44f.x 7/8 2020-09-12T13:54:08.332 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-diskprediction-local-2:16.0.0-5485.gf346cc7 8/8 2020-09-12T13:54:08.577 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.0.0-5485.gf346cc7 8/8 2020-09-12T13:54:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.0.0-5485.gf346cc7 1/8 2020-09-12T13:54:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-0.3.3-2.el8.x86_64 2/8 2020-09-12T13:54:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-threads-0.3.3-2.el8.x86_64 3/8 2020-09-12T13:54:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 4/8 2020-09-12T13:54:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 5/8 2020-09-12T13:54:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af44f.x 6/8 2020-09-12T13:54:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libgfortran-8.3.1-4.5.el8.x86_64 7/8 2020-09-12T13:54:08.852 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libquadmath-8.3.1-4.5.el8.x86_64 8/8 2020-09-12T13:54:08.852 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:08.852 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2020-09-12T13:54:08.853 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local-2:16.0.0-5485.gf346cc73d15.el8.noarch 2020-09-12T13:54:08.853 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-0.3.3-2.el8.x86_64 2020-09-12T13:54:08.853 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-threads-0.3.3-2.el8.x86_64 2020-09-12T13:54:08.853 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2020-09-12T13:54:08.853 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2020-09-12T13:54:08.853 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy-1.0.0-20.module_el8.1.0+245+c39af44f.x86_64 2020-09-12T13:54:08.853 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran-8.3.1-4.5.el8.x86_64 2020-09-12T13:54:08.854 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath-8.3.1-4.5.el8.x86_64 2020-09-12T13:54:08.854 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:08.854 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:54:08.966 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr-rook 2020-09-12T13:54:09.407 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:29 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:54:09.808 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:54:09.810 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:09.810 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2020-09-12T13:54:09.810 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:09.810 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2020-09-12T13:54:09.810 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook noarch 2:16.0.0-5485.gf346cc73d15.el8 2020-09-12T13:54:09.810 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-noarch 27 k 2020-09-12T13:54:09.811 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2020-09-12T13:54:09.811 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2020-09-12T13:54:09.811 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2020-09-12T13:54:09.811 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2020-09-12T13:54:09.811 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2020-09-12T13:54:09.811 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-Base 155 k 2020-09-12T13:54:09.811 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-Base 43 k 2020-09-12T13:54:09.811 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2020-09-12T13:54:09.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2020-09-12T13:54:09.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2020-09-12T13:54:09.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes noarch 11.0.0-2.el8 epel 1.2 M 2020-09-12T13:54:09.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa noarch 3.4.2-15.el8 epel 73 k 2020-09-12T13:54:09.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2020-09-12T13:54:09.812 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:09.812 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2020-09-12T13:54:09.813 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:09.813 INFO:teuthology.orchestra.run.smithi027.stdout:Install 13 Packages 2020-09-12T13:54:09.813 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:09.814 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 1.9 M 2020-09-12T13:54:09.814 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 24 M 2020-09-12T13:54:09.814 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2020-09-12T13:54:09.846 INFO:teuthology.orchestra.run.smithi027.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 824 kB/s | 27 kB 00:00 2020-09-12T13:54:09.849 INFO:teuthology.orchestra.run.smithi027.stdout:(2/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 40 MB/s | 126 kB 00:00 2020-09-12T13:54:09.851 INFO:teuthology.orchestra.run.smithi027.stdout:(3/13): python3-jsonpointer-1.10-11.el8.noarch. 533 kB/s | 20 kB 00:00 2020-09-12T13:54:09.852 INFO:teuthology.orchestra.run.smithi027.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 44 MB/s | 110 kB 00:00 2020-09-12T13:54:09.854 INFO:teuthology.orchestra.run.smithi027.stdout:(5/13): python3-requests-oauthlib-1.0.0-1.el8.n 18 MB/s | 43 kB 00:00 2020-09-12T13:54:09.854 INFO:teuthology.orchestra.run.smithi027.stdout:(6/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 41 MB/s | 155 kB 00:00 2020-09-12T13:54:09.903 INFO:teuthology.orchestra.run.smithi027.stdout:(7/13): python3-cachetools-3.1.1-4.el8.noarch.r 668 kB/s | 33 kB 00:00 2020-09-12T13:54:09.913 INFO:teuthology.orchestra.run.smithi027.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 280 kB/s | 16 kB 00:00 2020-09-12T13:54:09.936 INFO:teuthology.orchestra.run.smithi027.stdout:(9/13): python3-google-auth-1.1.1-10.el8.noarch 2.5 MB/s | 81 kB 00:00 2020-09-12T13:54:09.937 INFO:teuthology.orchestra.run.smithi027.stdout:(10/13): ceph-mgr-rook-16.0.0-5485.gf346cc73d15 219 kB/s | 27 kB 00:00 2020-09-12T13:54:09.953 INFO:teuthology.orchestra.run.smithi027.stdout:(11/13): python3-rsa-3.4.2-15.el8.noarch.rpm 4.3 MB/s | 73 kB 00:00 2020-09-12T13:54:10.014 INFO:teuthology.orchestra.run.smithi027.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 791 kB/s | 61 kB 00:00 2020-09-12T13:54:10.033 INFO:teuthology.orchestra.run.smithi027.stdout:(13/13): python3-kubernetes-11.0.0-2.el8.noarch 9.8 MB/s | 1.2 MB 00:00 2020-09-12T13:54:10.034 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2020-09-12T13:54:10.034 INFO:teuthology.orchestra.run.smithi027.stdout:Total 8.6 MB/s | 1.9 MB 00:00 2020-09-12T13:54:10.042 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2020-09-12T13:54:10.057 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2020-09-12T13:54:10.057 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2020-09-12T13:54:10.131 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2020-09-12T13:54:10.131 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2020-09-12T13:54:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2020-09-12T13:54:11.222 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2020-09-12T13:54:11.474 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2020-09-12T13:54:11.678 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rsa-3.4.2-15.el8.noarch 3/13 2020-09-12T13:54:11.864 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2020-09-12T13:54:12.040 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2020-09-12T13:54:12.276 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2020-09-12T13:54:12.548 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2020-09-12T13:54:12.781 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2020-09-12T13:54:13.870 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2020-09-12T13:54:14.147 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-kubernetes-11.0.0-2.el8.noarch 10/13 2020-09-12T13:54:14.305 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2020-09-12T13:54:14.524 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2020-09-12T13:54:14.698 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-rook-2:16.0.0-5485.gf346cc73d15.el8.noarc 13/13 2020-09-12T13:54:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-rook-2:16.0.0-5485.gf346cc73d15.el8.noarc 13/13 2020-09-12T13:54:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-rook-2:16.0.0-5485.gf346cc73d15.el8.noarc 1/13 2020-09-12T13:54:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2020-09-12T13:54:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2020-09-12T13:54:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2020-09-12T13:54:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2020-09-12T13:54:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2020-09-12T13:54:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2020-09-12T13:54:14.916 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2020-09-12T13:54:14.916 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2020-09-12T13:54:14.916 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2020-09-12T13:54:14.916 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-kubernetes-11.0.0-2.el8.noarch 11/13 2020-09-12T13:54:14.916 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rsa-3.4.2-15.el8.noarch 12/13 2020-09-12T13:54:15.260 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2020-09-12T13:54:15.260 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:15.261 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2020-09-12T13:54:15.261 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook-2:16.0.0-5485.gf346cc73d15.el8.noarch 2020-09-12T13:54:15.261 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch-1.21-2.el8.noarch 2020-09-12T13:54:15.262 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer-1.10-11.el8.noarch 2020-09-12T13:54:15.262 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2020-09-12T13:54:15.262 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2020-09-12T13:54:15.262 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2020-09-12T13:54:15.263 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2020-09-12T13:54:15.263 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools-3.1.1-4.el8.noarch 2020-09-12T13:54:15.263 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi-2018.10.15-7.el8.noarch 2020-09-12T13:54:15.263 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2020-09-12T13:54:15.264 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes-11.0.0-2.el8.noarch 2020-09-12T13:54:15.264 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa-3.4.2-15.el8.noarch 2020-09-12T13:54:15.264 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2020-09-12T13:54:15.264 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:15.267 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:54:15.390 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr-cephadm 2020-09-12T13:54:15.836 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:35 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:54:16.736 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:54:16.737 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:16.738 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2020-09-12T13:54:16.738 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:16.738 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2020-09-12T13:54:16.738 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm noarch 2:16.0.0-5485.gf346cc73d15.el8 ceph-noarch 63 k 2020-09-12T13:54:16.738 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2020-09-12T13:54:16.738 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel noarch 2.5.1-5.el8 CentOS-AppStream 4.8 M 2020-09-12T13:54:16.739 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 CentOS-AppStream 538 k 2020-09-12T13:54:16.739 INFO:teuthology.orchestra.run.smithi027.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2020-09-12T13:54:16.739 INFO:teuthology.orchestra.run.smithi027.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2020-09-12T13:54:16.739 INFO:teuthology.orchestra.run.smithi027.stdout: python3-remoto noarch 1.1.4-4.el8 epel 48 k 2020-09-12T13:54:16.739 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:16.739 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2020-09-12T13:54:16.739 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:16.739 INFO:teuthology.orchestra.run.smithi027.stdout:Install 6 Packages 2020-09-12T13:54:16.740 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:16.740 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 5.8 M 2020-09-12T13:54:16.740 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 24 M 2020-09-12T13:54:16.740 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2020-09-12T13:54:16.794 INFO:teuthology.orchestra.run.smithi027.stdout:(1/6): python3-jinja2-2.10.1-2.el8_0.noarch.rpm 9.8 MB/s | 538 kB 00:00 2020-09-12T13:54:16.855 INFO:teuthology.orchestra.run.smithi027.stdout:(2/6): python3-babel-2.5.1-5.el8.noarch.rpm 42 MB/s | 4.8 MB 00:00 2020-09-12T13:54:16.856 INFO:teuthology.orchestra.run.smithi027.stdout:(3/6): python3-apipkg-1.5-6.el8.noarch.rpm 345 kB/s | 21 kB 00:00 2020-09-12T13:54:16.887 INFO:teuthology.orchestra.run.smithi027.stdout:(4/6): ceph-mgr-cephadm-16.0.0-5485.gf346cc73d1 433 kB/s | 63 kB 00:00 2020-09-12T13:54:16.921 INFO:teuthology.orchestra.run.smithi027.stdout:(5/6): python3-execnet-1.7.1-1.el8.noarch.rpm 5.3 MB/s | 355 kB 00:00 2020-09-12T13:54:16.934 INFO:teuthology.orchestra.run.smithi027.stdout:(6/6): python3-remoto-1.1.4-4.el8.noarch.rpm 610 kB/s | 48 kB 00:00 2020-09-12T13:54:16.935 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2020-09-12T13:54:16.936 INFO:teuthology.orchestra.run.smithi027.stdout:Total 29 MB/s | 5.8 MB 00:00 2020-09-12T13:54:16.941 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2020-09-12T13:54:16.951 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2020-09-12T13:54:16.951 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2020-09-12T13:54:17.033 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2020-09-12T13:54:17.034 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2020-09-12T13:54:17.314 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2020-09-12T13:54:17.722 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 1/6 2020-09-12T13:54:17.961 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 2/6 2020-09-12T13:54:18.988 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-remoto-1.1.4-4.el8.noarch 3/6 2020-09-12T13:54:19.338 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 4/6 2020-09-12T13:54:19.615 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 5/6 2020-09-12T13:54:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-cephadm-2:16.0.0-5485.gf346cc73d15.el8.noar 6/6 2020-09-12T13:54:19.967 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.0.0-5485.gf346cc73d15.el8.noar 6/6 2020-09-12T13:54:19.967 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-cephadm-2:16.0.0-5485.gf346cc73d15.el8.noar 1/6 2020-09-12T13:54:19.967 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 2/6 2020-09-12T13:54:19.967 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 3/6 2020-09-12T13:54:19.968 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 4/6 2020-09-12T13:54:19.968 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 5/6 2020-09-12T13:54:20.234 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-remoto-1.1.4-4.el8.noarch 6/6 2020-09-12T13:54:20.234 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:20.234 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2020-09-12T13:54:20.234 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm-2:16.0.0-5485.gf346cc73d15.el8.noarch 2020-09-12T13:54:20.235 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel-2.5.1-5.el8.noarch 2020-09-12T13:54:20.235 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2020-09-12T13:54:20.235 INFO:teuthology.orchestra.run.smithi027.stdout: python3-apipkg-1.5-6.el8.noarch 2020-09-12T13:54:20.235 INFO:teuthology.orchestra.run.smithi027.stdout: python3-execnet-1.7.1-1.el8.noarch 2020-09-12T13:54:20.235 INFO:teuthology.orchestra.run.smithi027.stdout: python3-remoto-1.1.4-4.el8.noarch 2020-09-12T13:54:20.235 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:20.235 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:54:20.366 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-fuse 2020-09-12T13:54:20.815 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:40 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:54:21.196 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:54:21.197 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:21.197 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2020-09-12T13:54:21.197 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:21.197 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2020-09-12T13:54:21.198 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 623 k 2020-09-12T13:54:21.198 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2020-09-12T13:54:21.198 INFO:teuthology.orchestra.run.smithi027.stdout: fuse x86_64 2.9.7-12.el8 CentOS-Base 82 k 2020-09-12T13:54:21.198 INFO:teuthology.orchestra.run.smithi027.stdout: fuse-common x86_64 3.2.1-12.el8 CentOS-Base 21 k 2020-09-12T13:54:21.198 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:21.198 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2020-09-12T13:54:21.198 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:21.199 INFO:teuthology.orchestra.run.smithi027.stdout:Install 3 Packages 2020-09-12T13:54:21.199 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:21.199 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 727 k 2020-09-12T13:54:21.199 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 2.2 M 2020-09-12T13:54:21.200 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2020-09-12T13:54:21.232 INFO:teuthology.orchestra.run.smithi027.stdout:(1/3): fuse-2.9.7-12.el8.x86_64.rpm 2.5 MB/s | 82 kB 00:00 2020-09-12T13:54:21.232 INFO:teuthology.orchestra.run.smithi027.stdout:(2/3): fuse-common-3.2.1-12.el8.x86_64.rpm 636 kB/s | 21 kB 00:00 2020-09-12T13:54:21.415 INFO:teuthology.orchestra.run.smithi027.stdout:(3/3): ceph-fuse-16.0.0-5485.gf346cc73d15.el8.x 2.8 MB/s | 623 kB 00:00 2020-09-12T13:54:21.417 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2020-09-12T13:54:21.417 INFO:teuthology.orchestra.run.smithi027.stdout:Total 3.2 MB/s | 727 kB 00:00 2020-09-12T13:54:21.421 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2020-09-12T13:54:21.444 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2020-09-12T13:54:21.445 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2020-09-12T13:54:21.516 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2020-09-12T13:54:21.517 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2020-09-12T13:54:21.708 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2020-09-12T13:54:21.708 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fuse-common-3.2.1-12.el8.x86_64 1/3 2020-09-12T13:54:21.708 INFO:teuthology.orchestra.run.smithi027.stdout:warning: /etc/fuse.conf created as /etc/fuse.conf.rpmnew 2020-09-12T13:54:21.709 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:22.446 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fuse-2.9.7-12.el8.x86_64 2/3 2020-09-12T13:54:22.638 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-fuse-2:16.0.0-5485.gf346cc73d15.el8.x86_64 3/3 2020-09-12T13:54:22.958 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-fuse-2:16.0.0-5485.gf346cc73d15.el8.x86_64 3/3 2020-09-12T13:54:22.958 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-fuse-2:16.0.0-5485.gf346cc73d15.el8.x86_64 1/3 2020-09-12T13:54:22.958 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 2/3 2020-09-12T13:54:23.232 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-common-3.2.1-12.el8.x86_64 3/3 2020-09-12T13:54:23.233 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:23.233 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2020-09-12T13:54:23.233 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse-2:16.0.0-5485.gf346cc73d15.el8.x86_64 fuse-2.9.7-12.el8.x86_64 2020-09-12T13:54:23.233 INFO:teuthology.orchestra.run.smithi027.stdout: fuse-common-3.2.1-12.el8.x86_64 2020-09-12T13:54:23.233 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:23.233 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:54:23.525 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install librados-devel 2020-09-12T13:54:23.973 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:43 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:54:24.404 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:54:24.405 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:24.406 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repo Size 2020-09-12T13:54:24.406 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:24.406 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2020-09-12T13:54:24.406 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 131 k 2020-09-12T13:54:24.407 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:24.407 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2020-09-12T13:54:24.407 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:24.407 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2020-09-12T13:54:24.408 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:24.408 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 131 k 2020-09-12T13:54:24.408 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 436 k 2020-09-12T13:54:24.408 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2020-09-12T13:54:24.574 INFO:teuthology.orchestra.run.smithi027.stdout:librados-devel-16.0.0-5485.gf346cc73d15.el8.x86 778 kB/s | 131 kB 00:00 2020-09-12T13:54:24.575 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2020-09-12T13:54:24.575 INFO:teuthology.orchestra.run.smithi027.stdout:Total 773 kB/s | 131 kB 00:00 2020-09-12T13:54:24.577 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2020-09-12T13:54:24.584 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2020-09-12T13:54:24.584 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2020-09-12T13:54:24.638 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2020-09-12T13:54:24.639 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2020-09-12T13:54:24.752 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2020-09-12T13:54:24.970 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librados-devel-2:16.0.0-5485.gf346cc73d15.el8.x86_64 1/1 2020-09-12T13:54:25.172 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados-devel-2:16.0.0-5485.gf346cc73d15.el8.x86_64 1/1 2020-09-12T13:54:25.425 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados-devel-2:16.0.0-5485.gf346cc73d15.el8.x86_64 1/1 2020-09-12T13:54:25.426 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:25.426 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2020-09-12T13:54:25.426 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:54:25.426 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:25.426 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:54:25.724 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install libcephfs2 2020-09-12T13:54:26.171 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:46 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:54:26.445 INFO:teuthology.orchestra.run.smithi027.stdout:Package libcephfs2-2:16.0.0-5485.gf346cc73d15.el8.x86_64 is already installed. 2020-09-12T13:54:26.561 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:54:26.562 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2020-09-12T13:54:26.563 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:54:26.680 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install libcephfs-devel 2020-09-12T13:54:27.126 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:47 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:54:27.537 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:54:27.538 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:27.539 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repo Size 2020-09-12T13:54:27.539 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:27.539 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2020-09-12T13:54:27.540 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 23 k 2020-09-12T13:54:27.540 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:27.540 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2020-09-12T13:54:27.540 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:27.541 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2020-09-12T13:54:27.541 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:27.541 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 23 k 2020-09-12T13:54:27.542 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 84 k 2020-09-12T13:54:27.542 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2020-09-12T13:54:27.664 INFO:teuthology.orchestra.run.smithi027.stdout:libcephfs-devel-16.0.0-5485.gf346cc73d15.el8.x8 187 kB/s | 23 kB 00:00 2020-09-12T13:54:27.665 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2020-09-12T13:54:27.665 INFO:teuthology.orchestra.run.smithi027.stdout:Total 185 kB/s | 23 kB 00:00 2020-09-12T13:54:27.667 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2020-09-12T13:54:27.671 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2020-09-12T13:54:27.671 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2020-09-12T13:54:27.673 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2020-09-12T13:54:27.674 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2020-09-12T13:54:28.157 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2020-09-12T13:54:28.361 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs-devel-2:16.0.0-5485.gf346cc73d15.el8.x86_6 1/1 2020-09-12T13:54:28.570 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs-devel-2:16.0.0-5485.gf346cc73d15.el8.x86_6 1/1 2020-09-12T13:54:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs-devel-2:16.0.0-5485.gf346cc73d15.el8.x86_6 1/1 2020-09-12T13:54:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:28.800 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2020-09-12T13:54:28.800 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:54:28.800 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:28.800 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:54:29.070 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install librados2 2020-09-12T13:54:29.516 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:49 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:54:29.786 INFO:teuthology.orchestra.run.smithi027.stdout:Package librados2-2:16.0.0-5485.gf346cc73d15.el8.x86_64 is already installed. 2020-09-12T13:54:29.906 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:54:29.907 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2020-09-12T13:54:29.907 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:54:30.049 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install librbd1 2020-09-12T13:54:30.493 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:50 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:54:30.778 INFO:teuthology.orchestra.run.smithi027.stdout:Package librbd1-2:16.0.0-5485.gf346cc73d15.el8.x86_64 is already installed. 2020-09-12T13:54:30.896 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:54:30.897 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2020-09-12T13:54:30.897 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:54:31.021 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install python3-rados 2020-09-12T13:54:31.466 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:51 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:54:31.753 INFO:teuthology.orchestra.run.smithi027.stdout:Package python3-rados-2:16.0.0-5485.gf346cc73d15.el8.x86_64 is already installed. 2020-09-12T13:54:31.879 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:54:31.880 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2020-09-12T13:54:31.880 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:54:32.049 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install python3-rgw 2020-09-12T13:54:32.490 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:52 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:54:32.761 INFO:teuthology.orchestra.run.smithi027.stdout:Package python3-rgw-2:16.0.0-5485.gf346cc73d15.el8.x86_64 is already installed. 2020-09-12T13:54:32.882 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:54:32.883 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2020-09-12T13:54:32.883 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:54:33.005 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install python3-cephfs 2020-09-12T13:54:33.458 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:53 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:54:33.886 INFO:teuthology.orchestra.run.smithi027.stdout:Package python3-cephfs-2:16.0.0-5485.gf346cc73d15.el8.x86_64 is already installed. 2020-09-12T13:54:34.003 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:54:34.004 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2020-09-12T13:54:34.005 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:54:34.191 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install python3-rbd 2020-09-12T13:54:34.634 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:54 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:54:34.902 INFO:teuthology.orchestra.run.smithi027.stdout:Package python3-rbd-2:16.0.0-5485.gf346cc73d15.el8.x86_64 is already installed. 2020-09-12T13:54:35.023 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:54:35.024 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2020-09-12T13:54:35.024 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:54:35.165 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install rbd-fuse 2020-09-12T13:54:35.612 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:55 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:54:36.021 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:54:36.021 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:36.022 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repo Size 2020-09-12T13:54:36.022 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:36.022 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2020-09-12T13:54:36.022 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 87 k 2020-09-12T13:54:36.022 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:36.022 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2020-09-12T13:54:36.023 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:36.023 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2020-09-12T13:54:36.023 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:36.023 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 87 k 2020-09-12T13:54:36.023 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 292 k 2020-09-12T13:54:36.023 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2020-09-12T13:54:36.181 INFO:teuthology.orchestra.run.smithi027.stdout:rbd-fuse-16.0.0-5485.gf346cc73d15.el8.x86_64.rp 551 kB/s | 87 kB 00:00 2020-09-12T13:54:36.182 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2020-09-12T13:54:36.182 INFO:teuthology.orchestra.run.smithi027.stdout:Total 547 kB/s | 87 kB 00:00 2020-09-12T13:54:36.184 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2020-09-12T13:54:36.196 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2020-09-12T13:54:36.196 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2020-09-12T13:54:36.248 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2020-09-12T13:54:36.248 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2020-09-12T13:54:36.360 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2020-09-12T13:54:36.510 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-fuse-2:16.0.0-5485.gf346cc73d15.el8.x86_64 1/1 2020-09-12T13:54:36.710 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-fuse-2:16.0.0-5485.gf346cc73d15.el8.x86_64 1/1 2020-09-12T13:54:37.115 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-fuse-2:16.0.0-5485.gf346cc73d15.el8.x86_64 1/1 2020-09-12T13:54:37.115 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:37.115 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2020-09-12T13:54:37.116 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:54:37.116 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:37.116 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:54:37.449 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install rbd-mirror 2020-09-12T13:54:37.892 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:57 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:54:38.278 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:54:38.279 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:38.280 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repo Size 2020-09-12T13:54:38.280 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:38.280 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2020-09-12T13:54:38.280 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 3.1 M 2020-09-12T13:54:38.281 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:38.281 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2020-09-12T13:54:38.281 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:38.281 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2020-09-12T13:54:38.281 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:38.282 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 3.1 M 2020-09-12T13:54:38.282 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 14 M 2020-09-12T13:54:38.282 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2020-09-12T13:54:38.596 INFO:teuthology.orchestra.run.smithi027.stdout:rbd-mirror-16.0.0-5485.gf346cc73d15.el8.x86_64. 9.8 MB/s | 3.1 MB 00:00 2020-09-12T13:54:38.596 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2020-09-12T13:54:38.597 INFO:teuthology.orchestra.run.smithi027.stdout:Total 9.7 MB/s | 3.1 MB 00:00 2020-09-12T13:54:38.599 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2020-09-12T13:54:38.614 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2020-09-12T13:54:38.614 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2020-09-12T13:54:38.678 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2020-09-12T13:54:38.679 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2020-09-12T13:54:39.095 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2020-09-12T13:54:40.295 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-mirror-2:16.0.0-5485.gf346cc73d15.el8.x86_64 1/1 2020-09-12T13:54:40.620 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-mirror-2:16.0.0-5485.gf346cc73d15.el8.x86_64 1/1 2020-09-12T13:54:40.871 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-mirror-2:16.0.0-5485.gf346cc73d15.el8.x86_64 1/1 2020-09-12T13:54:40.871 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:40.871 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2020-09-12T13:54:40.872 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:54:40.872 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:40.872 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:54:41.162 INFO:teuthology.orchestra.run.smithi027:> sudo yum -y install rbd-nbd 2020-09-12T13:54:41.613 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:02:01 ago on Sat 12 Sep 2020 01:52:40 PM UTC. 2020-09-12T13:54:42.028 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2020-09-12T13:54:42.029 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:42.029 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repo Size 2020-09-12T13:54:42.029 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:42.029 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2020-09-12T13:54:42.029 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd x86_64 2:16.0.0-5485.gf346cc73d15.el8 ceph 161 k 2020-09-12T13:54:42.029 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:42.030 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2020-09-12T13:54:42.030 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2020-09-12T13:54:42.030 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2020-09-12T13:54:42.030 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:42.030 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 161 k 2020-09-12T13:54:42.030 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 608 k 2020-09-12T13:54:42.030 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2020-09-12T13:54:42.199 INFO:teuthology.orchestra.run.smithi027.stdout:rbd-nbd-16.0.0-5485.gf346cc73d15.el8.x86_64.rpm 950 kB/s | 161 kB 00:00 2020-09-12T13:54:42.200 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2020-09-12T13:54:42.200 INFO:teuthology.orchestra.run.smithi027.stdout:Total 943 kB/s | 161 kB 00:00 2020-09-12T13:54:42.202 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2020-09-12T13:54:42.216 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2020-09-12T13:54:42.216 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2020-09-12T13:54:42.270 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2020-09-12T13:54:42.270 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2020-09-12T13:54:42.396 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2020-09-12T13:54:42.609 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-nbd-2:16.0.0-5485.gf346cc73d15.el8.x86_64 1/1 2020-09-12T13:54:42.825 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-nbd-2:16.0.0-5485.gf346cc73d15.el8.x86_64 1/1 2020-09-12T13:54:43.130 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-nbd-2:16.0.0-5485.gf346cc73d15.el8.x86_64 1/1 2020-09-12T13:54:43.130 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:43.130 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2020-09-12T13:54:43.130 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd-2:16.0.0-5485.gf346cc73d15.el8.x86_64 2020-09-12T13:54:43.130 INFO:teuthology.orchestra.run.smithi027.stdout: 2020-09-12T13:54:43.131 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2020-09-12T13:54:43.438 DEBUG:teuthology.parallel:result is None 2020-09-12T13:54:43.439 INFO:teuthology.orchestra.run.smithi027:> sudo lsb_release -is 2020-09-12T13:54:43.480 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS 2020-09-12T13:54:43.481 DEBUG:teuthology.misc:System to be installed: CentOS 2020-09-12T13:54:43.481 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f346cc73d15a568d3bec1c99a58953de76f57b85 2020-09-12T13:54:43.660 INFO:teuthology.orchestra.run.smithi027:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2020-09-12T13:54:43.680 INFO:teuthology.orchestra.run.smithi027.stdout:16.0.0-5485.gf346cc73d15.el8 2020-09-12T13:54:43.681 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-5485.gf346cc73d15.el8 2020-09-12T13:54:43.682 INFO:teuthology.task.install:The correct ceph version 16.0.0-5485.gf346cc73d15 is installed. 2020-09-12T13:54:43.682 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2020-09-12T13:54:43.683 INFO:teuthology.orchestra.run.smithi027:> set -ex 2020-09-12T13:54:43.683 INFO:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-09-12T13:54:43.735 INFO:teuthology.orchestra.run.smithi027.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-09-12T13:54:43.751 INFO:teuthology.orchestra.run.smithi027.stderr:24+1 records in 2020-09-12T13:54:43.752 INFO:teuthology.orchestra.run.smithi027.stderr:24+1 records out 2020-09-12T13:54:43.752 INFO:teuthology.orchestra.run.smithi027.stderr:12667 bytes (13 kB, 12 KiB) copied, 8.2369e-05 s, 154 MB/s 2020-09-12T13:54:43.752 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2020-09-12T13:54:43.753 INFO:teuthology.orchestra.run.smithi027:> set -ex 2020-09-12T13:54:43.753 INFO:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/daemon-helper 2020-09-12T13:54:43.806 INFO:teuthology.orchestra.run.smithi027.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-09-12T13:54:43.822 INFO:teuthology.orchestra.run.smithi027.stderr:5+1 records in 2020-09-12T13:54:43.823 INFO:teuthology.orchestra.run.smithi027.stderr:5+1 records out 2020-09-12T13:54:43.823 INFO:teuthology.orchestra.run.smithi027.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 4.0957e-05 s, 74.7 MB/s 2020-09-12T13:54:43.824 INFO:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-09-12T13:54:43.894 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2020-09-12T13:54:43.895 INFO:teuthology.orchestra.run.smithi027:> set -ex 2020-09-12T13:54:43.895 INFO:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/adjust-ulimits 2020-09-12T13:54:43.948 INFO:teuthology.orchestra.run.smithi027.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-09-12T13:54:43.964 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records in 2020-09-12T13:54:43.964 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records out 2020-09-12T13:54:43.965 INFO:teuthology.orchestra.run.smithi027.stderr:299 bytes copied, 2.9525e-05 s, 10.1 MB/s 2020-09-12T13:54:43.965 INFO:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-09-12T13:54:44.037 INFO:teuthology.run_tasks:Running task workunit... 2020-09-12T13:54:44.055 INFO:tasks.workunit:Pulling workunits from ref f346cc73d15a568d3bec1c99a58953de76f57b85 2020-09-12T13:54:44.056 INFO:tasks.workunit:Making a separate scratch dir for every client... 2020-09-12T13:54:44.056 INFO:tasks.workunit:timeout=3h 2020-09-12T13:54:44.056 INFO:tasks.workunit:cleanup=True 2020-09-12T13:54:44.057 INFO:teuthology.orchestra.run.smithi027:> stat -- /home/ubuntu/cephtest/mnt.0 2020-09-12T13:54:44.091 INFO:teuthology.orchestra.run.smithi027.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-09-12T13:54:44.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-09-12T13:54:44.092 INFO:teuthology.orchestra.run.smithi027:> mkdir -- /home/ubuntu/cephtest/mnt.0 2020-09-12T13:54:44.145 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2020-09-12T13:54:44.146 INFO:teuthology.orchestra.run.smithi027:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2020-09-12T13:54:44.199 INFO:teuthology.orchestra.run.smithi027:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout f346cc73d15a568d3bec1c99a58953de76f57b85 2020-09-12T13:54:44.309 INFO:tasks.workunit.client.0.smithi027.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2020-09-12T13:55:35.414 INFO:tasks.workunit.client.0.smithi027.stderr:Note: checking out 'f346cc73d15a568d3bec1c99a58953de76f57b85'. 2020-09-12T13:55:35.415 INFO:tasks.workunit.client.0.smithi027.stderr: 2020-09-12T13:55:35.416 INFO:tasks.workunit.client.0.smithi027.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2020-09-12T13:55:35.416 INFO:tasks.workunit.client.0.smithi027.stderr:changes and commit them, and you can discard any commits you make in this 2020-09-12T13:55:35.417 INFO:tasks.workunit.client.0.smithi027.stderr:state without impacting any branches by performing another checkout. 2020-09-12T13:55:35.417 INFO:tasks.workunit.client.0.smithi027.stderr: 2020-09-12T13:55:35.418 INFO:tasks.workunit.client.0.smithi027.stderr:If you want to create a new branch to retain commits you create, you may 2020-09-12T13:55:35.418 INFO:tasks.workunit.client.0.smithi027.stderr:do so (now or later) by using -b with the checkout command again. Example: 2020-09-12T13:55:35.419 INFO:tasks.workunit.client.0.smithi027.stderr: 2020-09-12T13:55:35.420 INFO:tasks.workunit.client.0.smithi027.stderr: git checkout -b 2020-09-12T13:55:35.420 INFO:tasks.workunit.client.0.smithi027.stderr: 2020-09-12T13:55:35.421 INFO:tasks.workunit.client.0.smithi027.stderr:HEAD is now at f346cc73d15 Merge branch 'revert-32581-wip-djf-15070' of https://github.com/ceph/ceph into wip-kefu-testing-2020-09-12-2016 2020-09-12T13:55:35.426 INFO:teuthology.orchestra.run.smithi027:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2020-09-12T13:55:35.448 INFO:teuthology.orchestra.run.smithi027:> set -ex 2020-09-12T13:55:35.449 INFO:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2020-09-12T13:55:35.513 INFO:teuthology.orchestra.run.smithi027.stderr:+ dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2020-09-12T13:55:35.514 INFO:teuthology.orchestra.run.smithi027.stderr:2+1 records in 2020-09-12T13:55:35.514 INFO:teuthology.orchestra.run.smithi027.stderr:2+1 records out 2020-09-12T13:55:35.514 INFO:teuthology.orchestra.run.smithi027.stderr:1404 bytes (1.4 kB, 1.4 KiB) copied, 3.8761e-05 s, 36.2 MB/s 2020-09-12T13:55:35.515 INFO:teuthology.orchestra.run.smithi027.stdout:special/ceph_objectstore_tool.pyspecial/test-failure.shcrush/crush-classes.shcrush/crush-choose-args.shceph-helpers.shosd/osd-fast-mark-down.shosd/osd-recovery-prio.shosd/osd-backfill-prio.shosd/osd-dup.shosd/osd-copy-from.shosd/osd-reuse-id.shosd/osd-recovery-stats.shosd/osd-config.shosd/divergent-priors.shosd/pg-split-merge.shosd/osd-backfill-stats.shosd/repro_long_log.shosd/osd-reactivate.shosd/osd-backfill-space.shosd/ec-error-rollforward.shosd/osd-recovery-space.shosd/osd-backfill-recovery-log.shosd/repeer-on-acting-back.shosd/osd-bluefs-volume-ops.shosd/bad-inc-map.shosd/osd-force-create-pg.shosd/osd-bench.shosd/osd-rep-recov-eio.shosd/osd-markdown.shmisc/ok-to-stop.shmisc/network-ping.shmisc/test-ceph-helpers.shmisc/rados-striper.shmon/mon-scrub.shmon/mon-last-epoch-clean.shmon/osd-pool-df.shmon/test_pool_quota.shmon/osd-df.shmon/osd-pool-create.shmon/mkfs.shmon/health-mute.shmon/mon-created-time.shmon/osd-crush.shmon/osd-erasure-code-profile.shmon/misc.shmon/mon-bind.shmon/mon-osdmap-prune.shmon/mon-seesaw.shmon/mon-ping.shmon/mon-handle-forward.shscrub/osd-scrub-snaps.shscrub/osd-unexpected-clone.shscrub/osd-scrub-dump.shscrub/osd-recovery-scrub.shscrub/osd-scrub-repair.shscrub/osd-scrub-test.shmgr/balancer.sherasure-code/test-erasure-code-plugins.sherasure-code/test-erasure-eio.sherasure-code/test-erasure-code.sh 2020-09-12T13:55:35.516 INFO:tasks.workunit:Running workunits matching osd on client.0... 2020-09-12T13:55:35.517 INFO:tasks.workunit:Running workunit osd/bad-inc-map.sh... 2020-09-12T13:55:35.517 INFO:teuthology.orchestra.run.smithi027:workunit test osd/bad-inc-map.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f346cc73d15a568d3bec1c99a58953de76f57b85 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh 2020-09-12T13:55:35.587 INFO:tasks.workunit.client.0.smithi027.stderr:stty: 'standard input': Inappropriate ioctl for device 2020-09-12T13:55:35.618 INFO:tasks.workunit.client.0.smithi027.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2020-09-12T13:55:35.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2020-09-12T13:55:35.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2020-09-12T13:55:35.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2145: main: export PYTHONWARNINGS=ignore 2020-09-12T13:55:35.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2145: main: PYTHONWARNINGS=ignore 2020-09-12T13:55:35.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2146: main: export CEPH_CONF=/dev/null 2020-09-12T13:55:35.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2146: main: CEPH_CONF=/dev/null 2020-09-12T13:55:35.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2147: main: unset CEPH_ARGS 2020-09-12T13:55:35.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2149: main: local code 2020-09-12T13:55:35.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2150: main: run td/bad-inc-map 2020-09-12T13:55:35.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:8: run: local dir=td/bad-inc-map 2020-09-12T13:55:35.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:9: run: shift 2020-09-12T13:55:35.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:11: run: export CEPH_MON=127.0.0.1:41899 2020-09-12T13:55:35.621 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:11: run: CEPH_MON=127.0.0.1:41899 2020-09-12T13:55:35.621 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:12: run: export CEPH_ARGS 2020-09-12T13:55:35.621 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:13: run: uuidgen 2020-09-12T13:55:35.625 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:13: run: CEPH_ARGS+='--fsid=cc41d36c-23c6-4f70-87d7-088e9ce01134 --auth-supported=none ' 2020-09-12T13:55:35.625 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:14: run: CEPH_ARGS+='--mon-host=127.0.0.1:41899 ' 2020-09-12T13:55:35.625 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:15: run: set -e 2020-09-12T13:55:35.626 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:17: run: set 2020-09-12T13:55:35.626 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:17: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2020-09-12T13:55:35.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:17: run: local funcs=TEST_bad_inc_map 2020-09-12T13:55:35.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:18: run: for func in $funcs 2020-09-12T13:55:35.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:19: run: setup td/bad-inc-map 2020-09-12T13:55:35.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/bad-inc-map 2020-09-12T13:55:35.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/bad-inc-map 2020-09-12T13:55:35.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/bad-inc-map 2020-09-12T13:55:35.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T13:55:35.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/bad-inc-map KILL 2020-09-12T13:55:35.628 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T13:55:35.629 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T13:55:35.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T13:55:35.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T13:55:35.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T13:55:35.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T13:55:35.630 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T13:55:35.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T13:55:35.630 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T13:55:35.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T13:55:35.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T13:55:35.631 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T13:55:35.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T13:55:35.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T13:55:35.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T13:55:35.633 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T13:55:35.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T13:55:35.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T13:55:35.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/bad-inc-map 2020-09-12T13:55:35.635 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T13:55:35.635 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:55:35.636 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.17269 2020-09-12T13:55:35.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.17269 2020-09-12T13:55:35.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T13:55:35.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T13:55:35.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/bad-inc-map 2020-09-12T13:55:35.637 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2020-09-12T13:55:35.638 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:55:35.638 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.17269 2020-09-12T13:55:35.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.17269 2020-09-12T13:55:35.639 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2020-09-12T13:55:35.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2020-09-12T13:55:35.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2020-09-12T13:55:35.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2020-09-12T13:55:35.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/bad-inc-map 1' TERM HUP INT 2020-09-12T13:55:35.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:20: run: TEST_bad_inc_map td/bad-inc-map 2020-09-12T13:55:35.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:26: TEST_bad_inc_map: local dir=td/bad-inc-map 2020-09-12T13:55:35.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:28: TEST_bad_inc_map: run_mon td/bad-inc-map a 2020-09-12T13:55:35.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/bad-inc-map 2020-09-12T13:55:35.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2020-09-12T13:55:35.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2020-09-12T13:55:35.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2020-09-12T13:55:35.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/bad-inc-map/a 2020-09-12T13:55:35.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/bad-inc-map/a --run-dir=td/bad-inc-map 2020-09-12T13:55:36.005 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2020-09-12T13:55:36.005 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T13:55:36.005 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T13:55:36.006 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T13:55:36.006 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:55:36.006 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.17269 2020-09-12T13:55:36.006 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.17269/$cluster-$name.asok' 2020-09-12T13:55:36.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/bad-inc-map/a '--log-file=td/bad-inc-map/$name.log' '--admin-socket=/tmp/ceph-asok.17269/$cluster-$name.asok' --mon-cluster-log-file=td/bad-inc-map/log --run-dir=td/bad-inc-map '--pid-file=td/bad-inc-map/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 2020-09-12T13:55:36.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: cat 2020-09-12T13:55:36.175 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a fsid 2020-09-12T13:55:36.176 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T13:55:36.176 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T13:55:36.176 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=fsid 2020-09-12T13:55:36.176 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .fsid 2020-09-12T13:55:36.177 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T13:55:36.177 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T13:55:36.177 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T13:55:36.177 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T13:55:36.177 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:55:36.178 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.17269 2020-09-12T13:55:36.178 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.17269/ceph-mon.a.asok 2020-09-12T13:55:36.178 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T13:55:36.178 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.17269/ceph-mon.a.asok config get fsid 2020-09-12T13:55:36.294 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a mon_host 2020-09-12T13:55:36.294 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T13:55:36.294 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T13:55:36.294 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=mon_host 2020-09-12T13:55:36.294 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .mon_host 2020-09-12T13:55:36.295 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T13:55:36.295 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T13:55:36.295 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T13:55:36.295 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T13:55:36.295 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:55:36.295 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.17269 2020-09-12T13:55:36.295 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.17269/ceph-mon.a.asok 2020-09-12T13:55:36.296 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T13:55:36.296 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.17269/ceph-mon.a.asok config get mon_host 2020-09-12T13:55:36.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:29: TEST_bad_inc_map: run_mgr td/bad-inc-map x 2020-09-12T13:55:36.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: local dir=td/bad-inc-map 2020-09-12T13:55:36.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: shift 2020-09-12T13:55:36.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: local id=x 2020-09-12T13:55:36.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: shift 2020-09-12T13:55:36.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: local data=td/bad-inc-map/x 2020-09-12T13:55:36.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:558: run_mgr: ceph config set mgr mgr/devicehealth/enable_monitoring off --force 2020-09-12T13:55:36.874 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: get_asok_path 2020-09-12T13:55:36.874 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T13:55:36.874 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T13:55:36.874 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T13:55:36.875 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:55:36.875 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.17269 2020-09-12T13:55:36.875 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.17269/$cluster-$name.asok' 2020-09-12T13:55:36.875 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T13:55:36.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/bad-inc-map/x '--log-file=td/bad-inc-map/$name.log' '--admin-socket=/tmp/ceph-asok.17269/$cluster-$name.asok' --run-dir=td/bad-inc-map '--pid-file=td/bad-inc-map/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T13:55:36.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:30: TEST_bad_inc_map: run_osd td/bad-inc-map 0 2020-09-12T13:55:36.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/bad-inc-map 2020-09-12T13:55:36.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T13:55:36.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=0 2020-09-12T13:55:36.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T13:55:36.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/bad-inc-map/0 2020-09-12T13:55:36.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=cc41d36c-23c6-4f70-87d7-088e9ce01134 --auth-supported=none --mon-host=127.0.0.1:41899 ' 2020-09-12T13:55:36.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T13:55:36.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T13:55:36.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T13:55:36.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/bad-inc-map/0' 2020-09-12T13:55:36.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/bad-inc-map/0/journal' 2020-09-12T13:55:36.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T13:55:36.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T13:55:36.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/bad-inc-map' 2020-09-12T13:55:36.900 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T13:55:36.900 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T13:55:36.900 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T13:55:36.900 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T13:55:36.900 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:55:36.901 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.17269 2020-09-12T13:55:36.901 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.17269/$cluster-$name.asok' 2020-09-12T13:55:36.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.17269/$cluster-$name.asok' 2020-09-12T13:55:36.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T13:55:36.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T13:55:36.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T13:55:36.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/bad-inc-map/$name.log' 2020-09-12T13:55:36.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/bad-inc-map/$name.pid' 2020-09-12T13:55:36.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T13:55:36.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T13:55:36.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:55:36.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T13:55:36.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T13:55:36.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/bad-inc-map/0 2020-09-12T13:55:36.903 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T13:55:36.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=fd78fb46-9aed-4fa8-8912-87b8fdfeef25 2020-09-12T13:55:36.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd0 fd78fb46-9aed-4fa8-8912-87b8fdfeef25' 2020-09-12T13:55:36.903 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T13:55:36.904 INFO:tasks.workunit.client.0.smithi027.stdout:add osd0 fd78fb46-9aed-4fa8-8912-87b8fdfeef25 2020-09-12T13:55:36.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQBY01xfnX1UNhAAECsP4eDbPuTwVkpWAcKWIA== 2020-09-12T13:55:36.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQBY01xfnX1UNhAAECsP4eDbPuTwVkpWAcKWIA=="}' 2020-09-12T13:55:36.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new fd78fb46-9aed-4fa8-8912-87b8fdfeef25 -i td/bad-inc-map/0/new.json 2020-09-12T13:55:37.285 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T13:55:37.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/bad-inc-map/0/new.json 2020-09-12T13:55:37.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 0 --fsid=cc41d36c-23c6-4f70-87d7-088e9ce01134 --auth-supported=none --mon-host=127.0.0.1:41899 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/0 --osd-journal=td/bad-inc-map/0/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.17269/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQBY01xfnX1UNhAAECsP4eDbPuTwVkpWAcKWIA== --osd-uuid fd78fb46-9aed-4fa8-8912-87b8fdfeef25 2020-09-12T13:55:37.318 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:37.316+0000 7f424443eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:55:37.319 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:37.318+0000 7f424443eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:55:37.321 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:37.320+0000 7f424443eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:55:37.322 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:37.320+0000 7f424443eec0 -1 bluestore(td/bad-inc-map/0/block) _read_bdev_label failed to open td/bad-inc-map/0/block: (2) No such file or directory 2020-09-12T13:55:37.322 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:37.320+0000 7f424443eec0 -1 bluestore(td/bad-inc-map/0/block) _read_bdev_label failed to open td/bad-inc-map/0/block: (2) No such file or directory 2020-09-12T13:55:37.322 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:37.320+0000 7f424443eec0 -1 bluestore(td/bad-inc-map/0/block) _read_bdev_label failed to open td/bad-inc-map/0/block: (2) No such file or directory 2020-09-12T13:55:37.365 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:37.365+0000 7f424443eec0 -1 bluestore(td/bad-inc-map/0) _read_fsid unparsable uuid 2020-09-12T13:55:41.908 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/bad-inc-map/0/keyring 2020-09-12T13:55:41.908 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T13:55:41.909 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd0 key to auth repository 2020-09-12T13:55:41.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd0 key to auth repository 2020-09-12T13:55:41.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/bad-inc-map/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T13:55:42.186 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.0 2020-09-12T13:55:42.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.0 2020-09-12T13:55:42.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 0 --fsid=cc41d36c-23c6-4f70-87d7-088e9ce01134 --auth-supported=none --mon-host=127.0.0.1:41899 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/0 --osd-journal=td/bad-inc-map/0/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.17269/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:55:42.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T13:55:42.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T13:55:42.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T13:55:42.207 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:42.205+0000 7fed80205ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:55:42.208 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:42.207+0000 7fed80205ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:55:42.209 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:42.208+0000 7fed80205ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:55:42.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 0 2020-09-12T13:55:42.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T13:55:42.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T13:55:42.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T13:55:42.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T13:55:42.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:55:42.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T13:55:42.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:55:42.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T13:55:42.588 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T13:55:42.725 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:42.723+0000 7fed80205ec0 -1 Falling back to public interface 2020-09-12T13:55:42.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T13:55:43.528 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:43.526+0000 7fed80205ec0 -1 osd.0 0 log_to_monitors {default=true} 2020-09-12T13:55:43.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T13:55:43.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:55:43.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T13:55:43.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:55:43.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T13:55:43.984 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T13:55:44.413 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/18130,v1:127.0.0.1:6803/18130] [v2:127.0.0.1:6804/18130,v1:127.0.0.1:6805/18130] exists,up fd78fb46-9aed-4fa8-8912-87b8fdfeef25 2020-09-12T13:55:44.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T13:55:44.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T13:55:44.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T13:55:44.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:31: TEST_bad_inc_map: run_osd td/bad-inc-map 1 2020-09-12T13:55:44.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/bad-inc-map 2020-09-12T13:55:44.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T13:55:44.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=1 2020-09-12T13:55:44.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T13:55:44.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/bad-inc-map/1 2020-09-12T13:55:44.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=cc41d36c-23c6-4f70-87d7-088e9ce01134 --auth-supported=none --mon-host=127.0.0.1:41899 ' 2020-09-12T13:55:44.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T13:55:44.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T13:55:44.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T13:55:44.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/bad-inc-map/1' 2020-09-12T13:55:44.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/bad-inc-map/1/journal' 2020-09-12T13:55:44.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T13:55:44.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T13:55:44.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/bad-inc-map' 2020-09-12T13:55:44.416 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T13:55:44.416 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T13:55:44.417 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T13:55:44.417 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T13:55:44.417 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:55:44.417 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.17269 2020-09-12T13:55:44.417 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.17269/$cluster-$name.asok' 2020-09-12T13:55:44.417 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.17269/$cluster-$name.asok' 2020-09-12T13:55:44.417 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T13:55:44.417 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T13:55:44.418 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T13:55:44.418 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/bad-inc-map/$name.log' 2020-09-12T13:55:44.418 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/bad-inc-map/$name.pid' 2020-09-12T13:55:44.418 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T13:55:44.418 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T13:55:44.418 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:55:44.418 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T13:55:44.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T13:55:44.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/bad-inc-map/1 2020-09-12T13:55:44.419 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T13:55:44.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=03cd4a89-0c10-4548-a55f-fe7b83af9480 2020-09-12T13:55:44.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd1 03cd4a89-0c10-4548-a55f-fe7b83af9480' 2020-09-12T13:55:44.420 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T13:55:44.420 INFO:tasks.workunit.client.0.smithi027.stdout:add osd1 03cd4a89-0c10-4548-a55f-fe7b83af9480 2020-09-12T13:55:44.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQBg01xfmsRwGRAAK/eJmkba/9d9FZNSx+qrxQ== 2020-09-12T13:55:44.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQBg01xfmsRwGRAAK/eJmkba/9d9FZNSx+qrxQ=="}' 2020-09-12T13:55:44.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 03cd4a89-0c10-4548-a55f-fe7b83af9480 -i td/bad-inc-map/1/new.json 2020-09-12T13:55:44.964 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T13:55:44.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/bad-inc-map/1/new.json 2020-09-12T13:55:44.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 1 --fsid=cc41d36c-23c6-4f70-87d7-088e9ce01134 --auth-supported=none --mon-host=127.0.0.1:41899 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/1 --osd-journal=td/bad-inc-map/1/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.17269/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQBg01xfmsRwGRAAK/eJmkba/9d9FZNSx+qrxQ== --osd-uuid 03cd4a89-0c10-4548-a55f-fe7b83af9480 2020-09-12T13:55:44.998 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:44.996+0000 7f764c579ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:55:44.999 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:44.998+0000 7f764c579ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:55:45.000 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:44.999+0000 7f764c579ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:55:45.001 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:44.999+0000 7f764c579ec0 -1 bluestore(td/bad-inc-map/1/block) _read_bdev_label failed to open td/bad-inc-map/1/block: (2) No such file or directory 2020-09-12T13:55:45.001 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:44.999+0000 7f764c579ec0 -1 bluestore(td/bad-inc-map/1/block) _read_bdev_label failed to open td/bad-inc-map/1/block: (2) No such file or directory 2020-09-12T13:55:45.002 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:44.999+0000 7f764c579ec0 -1 bluestore(td/bad-inc-map/1/block) _read_bdev_label failed to open td/bad-inc-map/1/block: (2) No such file or directory 2020-09-12T13:55:45.125 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:45.123+0000 7f764c579ec0 -1 bluestore(td/bad-inc-map/1) _read_fsid unparsable uuid 2020-09-12T13:55:49.431 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/bad-inc-map/1/keyring 2020-09-12T13:55:49.431 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T13:55:49.432 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd1 key to auth repository 2020-09-12T13:55:49.433 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd1 key to auth repository 2020-09-12T13:55:49.433 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/bad-inc-map/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T13:55:49.968 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T13:55:49.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.1 2020-09-12T13:55:49.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T13:55:49.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T13:55:49.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T13:55:49.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 1 --fsid=cc41d36c-23c6-4f70-87d7-088e9ce01134 --auth-supported=none --mon-host=127.0.0.1:41899 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/1 --osd-journal=td/bad-inc-map/1/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.17269/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:55:49.988 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:49.986+0000 7fe04711bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:55:49.997 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:49.995+0000 7fe04711bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:55:49.998 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:49.996+0000 7fe04711bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:55:50.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 1 2020-09-12T13:55:50.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T13:55:50.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T13:55:50.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T13:55:50.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T13:55:50.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:55:50.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T13:55:50.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:55:50.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T13:55:50.370 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T13:55:50.514 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:50.512+0000 7fe04711bec0 -1 Falling back to public interface 2020-09-12T13:55:50.771 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T13:55:51.257 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:51.255+0000 7fe04711bec0 -1 osd.1 0 log_to_monitors {default=true} 2020-09-12T13:55:51.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T13:55:51.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:55:51.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T13:55:51.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:55:51.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T13:55:51.774 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T13:55:52.199 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/19124,v1:127.0.0.1:6811/19124] [v2:127.0.0.1:6812/19124,v1:127.0.0.1:6813/19124] exists,up 03cd4a89-0c10-4548-a55f-fe7b83af9480 2020-09-12T13:55:52.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T13:55:52.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T13:55:52.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T13:55:52.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:32: TEST_bad_inc_map: run_osd td/bad-inc-map 2 2020-09-12T13:55:52.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/bad-inc-map 2020-09-12T13:55:52.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T13:55:52.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=2 2020-09-12T13:55:52.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T13:55:52.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/bad-inc-map/2 2020-09-12T13:55:52.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=cc41d36c-23c6-4f70-87d7-088e9ce01134 --auth-supported=none --mon-host=127.0.0.1:41899 ' 2020-09-12T13:55:52.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T13:55:52.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T13:55:52.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T13:55:52.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/bad-inc-map/2' 2020-09-12T13:55:52.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/bad-inc-map/2/journal' 2020-09-12T13:55:52.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T13:55:52.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T13:55:52.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/bad-inc-map' 2020-09-12T13:55:52.202 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T13:55:52.202 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T13:55:52.202 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T13:55:52.203 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T13:55:52.203 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:55:52.203 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.17269 2020-09-12T13:55:52.203 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.17269/$cluster-$name.asok' 2020-09-12T13:55:52.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.17269/$cluster-$name.asok' 2020-09-12T13:55:52.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T13:55:52.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T13:55:52.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T13:55:52.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/bad-inc-map/$name.log' 2020-09-12T13:55:52.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/bad-inc-map/$name.pid' 2020-09-12T13:55:52.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T13:55:52.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T13:55:52.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:55:52.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T13:55:52.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T13:55:52.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/bad-inc-map/2 2020-09-12T13:55:52.206 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T13:55:52.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=4101bc2b-35b5-4945-a75f-cc91a8af964f 2020-09-12T13:55:52.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd2 4101bc2b-35b5-4945-a75f-cc91a8af964f' 2020-09-12T13:55:52.207 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T13:55:52.207 INFO:tasks.workunit.client.0.smithi027.stdout:add osd2 4101bc2b-35b5-4945-a75f-cc91a8af964f 2020-09-12T13:55:52.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQBo01xf5knQDBAA/+hGnpReIs2SPh2rQxim/w== 2020-09-12T13:55:52.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQBo01xf5knQDBAA/+hGnpReIs2SPh2rQxim/w=="}' 2020-09-12T13:55:52.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 4101bc2b-35b5-4945-a75f-cc91a8af964f -i td/bad-inc-map/2/new.json 2020-09-12T13:55:52.737 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T13:55:52.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/bad-inc-map/2/new.json 2020-09-12T13:55:52.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 2 --fsid=cc41d36c-23c6-4f70-87d7-088e9ce01134 --auth-supported=none --mon-host=127.0.0.1:41899 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/2 --osd-journal=td/bad-inc-map/2/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.17269/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQBo01xf5knQDBAA/+hGnpReIs2SPh2rQxim/w== --osd-uuid 4101bc2b-35b5-4945-a75f-cc91a8af964f 2020-09-12T13:55:52.770 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:52.768+0000 7fc65e1fcec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:55:52.771 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:52.770+0000 7fc65e1fcec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:55:52.772 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:52.770+0000 7fc65e1fcec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:55:52.773 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:52.771+0000 7fc65e1fcec0 -1 bluestore(td/bad-inc-map/2/block) _read_bdev_label failed to open td/bad-inc-map/2/block: (2) No such file or directory 2020-09-12T13:55:52.773 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:52.771+0000 7fc65e1fcec0 -1 bluestore(td/bad-inc-map/2/block) _read_bdev_label failed to open td/bad-inc-map/2/block: (2) No such file or directory 2020-09-12T13:55:52.773 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:52.771+0000 7fc65e1fcec0 -1 bluestore(td/bad-inc-map/2/block) _read_bdev_label failed to open td/bad-inc-map/2/block: (2) No such file or directory 2020-09-12T13:55:52.848 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:52.847+0000 7fc65e1fcec0 -1 bluestore(td/bad-inc-map/2) _read_fsid unparsable uuid 2020-09-12T13:55:57.422 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/bad-inc-map/2/keyring 2020-09-12T13:55:57.423 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T13:55:57.423 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd2 key to auth repository 2020-09-12T13:55:57.423 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/bad-inc-map/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T13:55:57.424 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd2 key to auth repository 2020-09-12T13:55:58.016 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.2 2020-09-12T13:55:58.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.2 2020-09-12T13:55:58.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 2 --fsid=cc41d36c-23c6-4f70-87d7-088e9ce01134 --auth-supported=none --mon-host=127.0.0.1:41899 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/2 --osd-journal=td/bad-inc-map/2/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.17269/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:55:58.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T13:55:58.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T13:55:58.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T13:55:58.036 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:58.035+0000 7fd7212f1ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:55:58.038 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:58.037+0000 7fd7212f1ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:55:58.039 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:58.038+0000 7fd7212f1ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:55:58.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 2 2020-09-12T13:55:58.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T13:55:58.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T13:55:58.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T13:55:58.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T13:55:58.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:55:58.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T13:55:58.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:55:58.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T13:55:58.416 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T13:55:58.555 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:58.555+0000 7fd7212f1ec0 -1 Falling back to public interface 2020-09-12T13:55:58.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T13:55:59.473 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:55:59.472+0000 7fd7212f1ec0 -1 osd.2 0 log_to_monitors {default=true} 2020-09-12T13:55:59.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T13:55:59.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:55:59.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T13:55:59.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:55:59.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T13:55:59.979 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T13:56:00.457 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/20126,v1:127.0.0.1:6819/20126] [v2:127.0.0.1:6820/20126,v1:127.0.0.1:6821/20126] exists,up 4101bc2b-35b5-4945-a75f-cc91a8af964f 2020-09-12T13:56:00.457 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T13:56:00.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T13:56:00.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T13:56:00.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:34: TEST_bad_inc_map: ceph config set osd.2 osd_inject_bad_map_crc_probability 1 2020-09-12T13:56:00.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:37: TEST_bad_inc_map: create_pool foo 8 2020-09-12T13:56:00.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create foo 8 2020-09-12T13:56:01.719 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'foo' already exists 2020-09-12T13:56:01.728 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T13:56:02.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:38: TEST_bad_inc_map: ceph osd pool set foo min_size 1 2020-09-12T13:56:03.664 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 1 min_size to 1 2020-09-12T13:56:03.673 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:39: TEST_bad_inc_map: ceph osd pool set foo min_size 2 2020-09-12T13:56:04.689 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 1 min_size to 2 2020-09-12T13:56:04.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:41: TEST_bad_inc_map: sleep 5 2020-09-12T13:56:09.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:44: TEST_bad_inc_map: TIMEOUT=10 2020-09-12T13:56:09.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:44: TEST_bad_inc_map: wait_for_osd up 0 2020-09-12T13:56:09.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T13:56:09.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T13:56:09.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T13:56:09.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T13:56:09.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 10 )) 2020-09-12T13:56:09.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T13:56:09.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:56:09.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T13:56:09.702 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T13:56:10.484 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 5 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/18130,v1:127.0.0.1:6803/18130] [v2:127.0.0.1:6804/18130,v1:127.0.0.1:6805/18130] exists,up fd78fb46-9aed-4fa8-8912-87b8fdfeef25 2020-09-12T13:56:10.485 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T13:56:10.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T13:56:10.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T13:56:10.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T13:56:10.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:45: TEST_bad_inc_map: TIMEOUT=10 2020-09-12T13:56:10.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:45: TEST_bad_inc_map: wait_for_osd up 1 2020-09-12T13:56:10.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T13:56:10.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T13:56:10.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T13:56:10.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T13:56:10.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 10 )) 2020-09-12T13:56:10.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T13:56:10.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:56:10.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T13:56:10.882 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 10 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/19124,v1:127.0.0.1:6811/19124] [v2:127.0.0.1:6812/19124,v1:127.0.0.1:6813/19124] exists,up 03cd4a89-0c10-4548-a55f-fe7b83af9480 2020-09-12T13:56:10.882 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T13:56:10.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T13:56:10.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T13:56:10.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T13:56:10.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:46: TEST_bad_inc_map: TIMEOUT=10 2020-09-12T13:56:10.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:46: TEST_bad_inc_map: wait_for_osd up 2 2020-09-12T13:56:10.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T13:56:10.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T13:56:10.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T13:56:10.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T13:56:10.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 10 )) 2020-09-12T13:56:10.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T13:56:10.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:56:10.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T13:56:11.285 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up in weight 1 up_from 15 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/20126,v1:127.0.0.1:6819/20126] [v2:127.0.0.1:6820/20126,v1:127.0.0.1:6821/20126] exists,up 4101bc2b-35b5-4945-a75f-cc91a8af964f 2020-09-12T13:56:11.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T13:56:11.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T13:56:11.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T13:56:11.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:49: TEST_bad_inc_map: grep 'injecting map crc failure' td/bad-inc-map/osd.2.log 2020-09-12T13:56:11.287 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:01.446+0000 7fd709a09700 -1 osd.2 15 handle_osd_map injecting map crc failure 2020-09-12T13:56:11.287 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:01.715+0000 7fd709a09700 -1 osd.2 17 handle_osd_map injecting map crc failure 2020-09-12T13:56:11.288 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:02.041+0000 7fd709a09700 -1 osd.2 18 handle_osd_map injecting map crc failure 2020-09-12T13:56:11.288 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:02.503+0000 7fd709a09700 -1 osd.2 19 handle_osd_map injecting map crc failure 2020-09-12T13:56:11.288 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:03.384+0000 7fd707a05700 -1 osd.2 20 handle_osd_map injecting map crc failure 2020-09-12T13:56:11.288 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:03.384+0000 7fd707a05700 -1 osd.2 20 handle_osd_map injecting map crc failure 2020-09-12T13:56:11.288 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:03.660+0000 7fd709a09700 -1 osd.2 23 handle_osd_map injecting map crc failure 2020-09-12T13:56:11.288 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:03.984+0000 7fd709a09700 -1 osd.2 24 handle_osd_map injecting map crc failure 2020-09-12T13:56:11.288 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:03.985+0000 7fd709a09700 -1 osd.2 24 handle_osd_map injecting map crc failure 2020-09-12T13:56:11.289 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:04.349+0000 7fd709a09700 -1 osd.2 25 handle_osd_map injecting map crc failure 2020-09-12T13:56:11.289 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:04.910+0000 7fd707a05700 -1 osd.2 26 handle_osd_map injecting map crc failure 2020-09-12T13:56:11.289 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:04.911+0000 7fd707a05700 -1 osd.2 26 handle_osd_map injecting map crc failure 2020-09-12T13:56:11.289 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:05.029+0000 7fd709a09700 -1 osd.2 26 handle_osd_map injecting map crc failure 2020-09-12T13:56:11.289 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:05.304+0000 7fd709a09700 -1 osd.2 28 handle_osd_map injecting map crc failure 2020-09-12T13:56:11.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:50: TEST_bad_inc_map: grep 'bailing because last' td/bad-inc-map/osd.2.log 2020-09-12T13:56:11.290 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:01.447+0000 7fd709a09700 10 osd.2 15 handle_osd_map bailing because last < start (15<16) 2020-09-12T13:56:11.290 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:01.716+0000 7fd709a09700 10 osd.2 17 handle_osd_map bailing because last < start (17<18) 2020-09-12T13:56:11.291 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:02.042+0000 7fd709a09700 10 osd.2 18 handle_osd_map bailing because last < start (18<19) 2020-09-12T13:56:11.291 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:02.503+0000 7fd709a09700 10 osd.2 19 handle_osd_map bailing because last < start (19<20) 2020-09-12T13:56:11.291 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:03.384+0000 7fd707a05700 10 osd.2 20 handle_osd_map bailing because last < start (20<21) 2020-09-12T13:56:11.291 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:03.385+0000 7fd707a05700 10 osd.2 20 handle_osd_map bailing because last < start (20<21) 2020-09-12T13:56:11.291 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:03.660+0000 7fd709a09700 10 osd.2 23 handle_osd_map bailing because last < start (23<24) 2020-09-12T13:56:11.292 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:03.984+0000 7fd709a09700 10 osd.2 24 handle_osd_map bailing because last < start (24<25) 2020-09-12T13:56:11.292 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:03.985+0000 7fd709a09700 10 osd.2 24 handle_osd_map bailing because last < start (24<25) 2020-09-12T13:56:11.292 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:04.350+0000 7fd709a09700 10 osd.2 25 handle_osd_map bailing because last < start (25<26) 2020-09-12T13:56:11.292 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:04.911+0000 7fd707a05700 10 osd.2 26 handle_osd_map bailing because last < start (26<27) 2020-09-12T13:56:11.292 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:04.911+0000 7fd707a05700 10 osd.2 26 handle_osd_map bailing because last < start (26<27) 2020-09-12T13:56:11.292 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:05.029+0000 7fd709a09700 10 osd.2 26 handle_osd_map bailing because last < start (27<28) 2020-09-12T13:56:11.292 INFO:tasks.workunit.client.0.smithi027.stdout:2020-09-12T13:56:05.305+0000 7fd709a09700 10 osd.2 28 handle_osd_map bailing because last < start (28<29) 2020-09-12T13:56:11.293 INFO:tasks.workunit.client.0.smithi027.stdout:success 2020-09-12T13:56:11.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:52: TEST_bad_inc_map: echo success 2020-09-12T13:56:11.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:54: TEST_bad_inc_map: delete_pool foo 2020-09-12T13:56:11.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: delete_pool: local poolname=foo 2020-09-12T13:56:11.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: ceph osd pool delete foo foo --yes-i-really-really-mean-it 2020-09-12T13:56:11.812 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'foo' does not exist 2020-09-12T13:56:11.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:55: TEST_bad_inc_map: kill_daemons td/bad-inc-map 2020-09-12T13:56:11.822 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T13:56:11.822 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T13:56:11.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T13:56:11.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T13:56:11.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T13:56:11.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T13:56:11.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:21: run: teardown td/bad-inc-map 2020-09-12T13:56:11.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/bad-inc-map 2020-09-12T13:56:11.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T13:56:11.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/bad-inc-map KILL 2020-09-12T13:56:11.931 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T13:56:11.931 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T13:56:11.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T13:56:11.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T13:56:11.932 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T13:56:11.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T13:56:11.935 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T13:56:11.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T13:56:11.936 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T13:56:11.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T13:56:11.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T13:56:11.937 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T13:56:11.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T13:56:11.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T13:56:11.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T13:56:11.940 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T13:56:11.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T13:56:11.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T13:56:11.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/bad-inc-map 2020-09-12T13:56:11.946 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T13:56:11.946 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:56:11.946 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.17269 2020-09-12T13:56:11.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.17269 2020-09-12T13:56:11.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T13:56:11.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T13:56:11.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2151: main: code=0 2020-09-12T13:56:11.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2155: main: teardown td/bad-inc-map 0 2020-09-12T13:56:11.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/bad-inc-map 2020-09-12T13:56:11.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2020-09-12T13:56:11.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/bad-inc-map KILL 2020-09-12T13:56:11.948 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T13:56:11.948 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T13:56:11.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T13:56:11.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T13:56:11.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T13:56:11.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T13:56:11.949 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T13:56:11.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T13:56:11.950 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T13:56:11.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T13:56:11.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T13:56:11.951 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T13:56:11.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T13:56:11.953 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T13:56:11.953 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T13:56:11.953 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T13:56:11.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T13:56:11.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2020-09-12T13:56:11.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/bad-inc-map 2020-09-12T13:56:11.955 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T13:56:11.956 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:56:11.956 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.17269 2020-09-12T13:56:11.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.17269 2020-09-12T13:56:11.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T13:56:11.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T13:56:11.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2156: main: return 0 2020-09-12T13:56:11.957 INFO:teuthology.orchestra.run:Running command with timeout 3600 2020-09-12T13:56:11.958 INFO:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2020-09-12T13:56:12.029 INFO:tasks.workunit:Running workunit osd/divergent-priors.sh... 2020-09-12T13:56:12.029 INFO:teuthology.orchestra.run.smithi027:workunit test osd/divergent-priors.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f346cc73d15a568d3bec1c99a58953de76f57b85 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh 2020-09-12T13:56:12.089 INFO:tasks.workunit.client.0.smithi027.stderr:stty: 'standard input': Inappropriate ioctl for device 2020-09-12T13:56:12.098 INFO:tasks.workunit.client.0.smithi027.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2020-09-12T13:56:12.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2020-09-12T13:56:12.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2020-09-12T13:56:12.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2145: main: export PYTHONWARNINGS=ignore 2020-09-12T13:56:12.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2145: main: PYTHONWARNINGS=ignore 2020-09-12T13:56:12.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2146: main: export CEPH_CONF=/dev/null 2020-09-12T13:56:12.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2146: main: CEPH_CONF=/dev/null 2020-09-12T13:56:12.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2147: main: unset CEPH_ARGS 2020-09-12T13:56:12.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2149: main: local code 2020-09-12T13:56:12.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2150: main: run td/divergent-priors 2020-09-12T13:56:12.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:21: run: local dir=td/divergent-priors 2020-09-12T13:56:12.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:22: run: shift 2020-09-12T13:56:12.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:25: run: export loglen=12 2020-09-12T13:56:12.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:25: run: loglen=12 2020-09-12T13:56:12.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:26: run: export divisor=3 2020-09-12T13:56:12.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:26: run: divisor=3 2020-09-12T13:56:12.101 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:27: run: expr 12 / 2 2020-09-12T13:56:12.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:27: run: export trim=6 2020-09-12T13:56:12.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:27: run: trim=6 2020-09-12T13:56:12.101 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:28: run: expr 6 / 3 2020-09-12T13:56:12.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:28: run: export DIVERGENT_WRITE=2 2020-09-12T13:56:12.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:28: run: DIVERGENT_WRITE=2 2020-09-12T13:56:12.103 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:29: run: expr 6 / 3 2020-09-12T13:56:12.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:29: run: export DIVERGENT_REMOVE=2 2020-09-12T13:56:12.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:29: run: DIVERGENT_REMOVE=2 2020-09-12T13:56:12.104 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:30: run: expr 6 / 3 2020-09-12T13:56:12.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:30: run: export DIVERGENT_CREATE=2 2020-09-12T13:56:12.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:30: run: DIVERGENT_CREATE=2 2020-09-12T13:56:12.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:31: run: export poolname=test 2020-09-12T13:56:12.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:31: run: poolname=test 2020-09-12T13:56:12.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:32: run: export testobjects=100 2020-09-12T13:56:12.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:32: run: testobjects=100 2020-09-12T13:56:12.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:34: run: export CEPH_MON=127.0.0.1:7115 2020-09-12T13:56:12.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:34: run: CEPH_MON=127.0.0.1:7115 2020-09-12T13:56:12.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:35: run: export CEPH_ARGS 2020-09-12T13:56:12.105 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:36: run: uuidgen 2020-09-12T13:56:12.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:36: run: CEPH_ARGS+='--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none ' 2020-09-12T13:56:12.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:37: run: CEPH_ARGS+='--mon-host=127.0.0.1:7115 ' 2020-09-12T13:56:12.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:39: run: CEPH_ARGS+='--osd_force_auth_primary_missing_objects=1000000 ' 2020-09-12T13:56:12.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:40: run: CEPH_ARGS+='--osd_debug_pg_log_writeout=true ' 2020-09-12T13:56:12.106 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:41: run: CEPH_ARGS+='--osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T13:56:12.106 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:43: run: set 2020-09-12T13:56:12.106 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:43: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2020-09-12T13:56:12.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:43: run: local 'funcs=TEST_divergent 2020-09-12T13:56:12.107 INFO:tasks.workunit.client.0.smithi027.stderr:TEST_divergent_2 2020-09-12T13:56:12.108 INFO:tasks.workunit.client.0.smithi027.stderr:TEST_divergent_3 2020-09-12T13:56:12.108 INFO:tasks.workunit.client.0.smithi027.stderr:TEST_divergent_ec' 2020-09-12T13:56:12.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2020-09-12T13:56:12.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2020-09-12T13:56:12.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/divergent-priors 2020-09-12T13:56:12.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/divergent-priors 2020-09-12T13:56:12.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2020-09-12T13:56:12.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T13:56:12.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2020-09-12T13:56:12.109 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T13:56:12.109 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T13:56:12.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T13:56:12.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T13:56:12.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T13:56:12.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T13:56:12.110 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T13:56:12.110 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T13:56:12.110 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T13:56:12.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T13:56:12.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T13:56:12.111 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T13:56:12.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T13:56:12.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T13:56:12.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T13:56:12.112 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T13:56:12.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T13:56:12.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T13:56:12.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2020-09-12T13:56:12.113 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T13:56:12.113 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:56:12.113 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:56:12.113 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.20815 2020-09-12T13:56:12.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T13:56:12.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T13:56:12.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/divergent-priors 2020-09-12T13:56:12.115 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2020-09-12T13:56:12.115 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:56:12.115 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:56:12.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.20815 2020-09-12T13:56:12.116 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2020-09-12T13:56:12.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2020-09-12T13:56:12.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2020-09-12T13:56:12.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2020-09-12T13:56:12.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2020-09-12T13:56:12.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent td/divergent-priors 2020-09-12T13:56:12.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:57: TEST_divergent: local dir=td/divergent-priors 2020-09-12T13:56:12.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:60: TEST_divergent: local dummyfile=/etc/fstab 2020-09-12T13:56:12.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:61: TEST_divergent: local dummyfile2=/etc/resolv.conf 2020-09-12T13:56:12.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:63: TEST_divergent: local num_osds=3 2020-09-12T13:56:12.118 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:64: TEST_divergent: expr 3 - 1 2020-09-12T13:56:12.119 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:64: TEST_divergent: seq 0 2 2020-09-12T13:56:12.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:64: TEST_divergent: local 'osds=0 2020-09-12T13:56:12.124 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T13:56:12.124 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T13:56:12.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:65: TEST_divergent: run_mon td/divergent-priors a 2020-09-12T13:56:12.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/divergent-priors 2020-09-12T13:56:12.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2020-09-12T13:56:12.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2020-09-12T13:56:12.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2020-09-12T13:56:12.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/divergent-priors/a 2020-09-12T13:56:12.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2020-09-12T13:56:12.440 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2020-09-12T13:56:12.440 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T13:56:12.440 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T13:56:12.440 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T13:56:12.441 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:56:12.441 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:56:12.441 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:56:12.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 2020-09-12T13:56:12.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: cat 2020-09-12T13:56:12.643 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a fsid 2020-09-12T13:56:12.644 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T13:56:12.644 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T13:56:12.644 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=fsid 2020-09-12T13:56:12.644 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .fsid 2020-09-12T13:56:12.644 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T13:56:12.644 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T13:56:12.644 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T13:56:12.645 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T13:56:12.645 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:56:12.645 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:56:12.645 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-mon.a.asok 2020-09-12T13:56:12.645 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T13:56:12.646 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.20815/ceph-mon.a.asok config get fsid 2020-09-12T13:56:12.749 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a mon_host 2020-09-12T13:56:12.749 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T13:56:12.749 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T13:56:12.749 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=mon_host 2020-09-12T13:56:12.750 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .mon_host 2020-09-12T13:56:12.750 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T13:56:12.750 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T13:56:12.750 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T13:56:12.750 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T13:56:12.750 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:56:12.750 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:56:12.751 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-mon.a.asok 2020-09-12T13:56:12.751 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T13:56:12.751 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.20815/ceph-mon.a.asok config get mon_host 2020-09-12T13:56:12.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:66: TEST_divergent: run_mgr td/divergent-priors x 2020-09-12T13:56:12.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: local dir=td/divergent-priors 2020-09-12T13:56:12.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: shift 2020-09-12T13:56:12.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: local id=x 2020-09-12T13:56:12.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: shift 2020-09-12T13:56:12.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: local data=td/divergent-priors/x 2020-09-12T13:56:12.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:558: run_mgr: ceph config set mgr mgr/devicehealth/enable_monitoring off --force 2020-09-12T13:56:13.786 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: get_asok_path 2020-09-12T13:56:13.786 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T13:56:13.787 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T13:56:13.787 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T13:56:13.787 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:56:13.787 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:56:13.787 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:56:13.788 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T13:56:13.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T13:56:13.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:67: TEST_divergent: for i in $osds 2020-09-12T13:56:13.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:69: TEST_divergent: run_osd td/divergent-priors 0 2020-09-12T13:56:13.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/divergent-priors 2020-09-12T13:56:13.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T13:56:13.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=0 2020-09-12T13:56:13.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T13:56:13.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/divergent-priors/0 2020-09-12T13:56:13.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T13:56:13.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T13:56:13.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T13:56:13.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T13:56:13.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2020-09-12T13:56:13.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2020-09-12T13:56:13.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T13:56:13.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T13:56:13.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T13:56:13.819 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T13:56:13.819 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T13:56:13.819 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T13:56:13.819 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T13:56:13.819 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:56:13.819 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:56:13.820 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:56:13.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:56:13.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T13:56:13.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T13:56:13.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T13:56:13.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T13:56:13.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T13:56:13.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T13:56:13.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T13:56:13.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:56:13.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T13:56:13.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T13:56:13.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/divergent-priors/0 2020-09-12T13:56:13.822 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T13:56:13.823 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=97c8d704-849c-44e1-a64d-c66c37f051bf 2020-09-12T13:56:13.823 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd0 97c8d704-849c-44e1-a64d-c66c37f051bf' 2020-09-12T13:56:13.823 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T13:56:13.823 INFO:tasks.workunit.client.0.smithi027.stdout:add osd0 97c8d704-849c-44e1-a64d-c66c37f051bf 2020-09-12T13:56:13.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQB901xfMMnUMRAAvOMuS756WqO5SwbmKdiFEA== 2020-09-12T13:56:13.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQB901xfMMnUMRAAvOMuS756WqO5SwbmKdiFEA=="}' 2020-09-12T13:56:13.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 97c8d704-849c-44e1-a64d-c66c37f051bf -i td/divergent-priors/0/new.json 2020-09-12T13:56:14.202 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T13:56:14.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/divergent-priors/0/new.json 2020-09-12T13:56:14.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 0 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQB901xfMMnUMRAAvOMuS756WqO5SwbmKdiFEA== --osd-uuid 97c8d704-849c-44e1-a64d-c66c37f051bf 2020-09-12T13:56:14.245 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:14.244+0000 7f5da8b89ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:56:14.246 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:14.246+0000 7f5da8b89ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:56:14.247 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:14.247+0000 7f5da8b89ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:56:14.248 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:14.248+0000 7f5da8b89ec0 -1 bluestore(td/divergent-priors/0/block) _read_bdev_label failed to open td/divergent-priors/0/block: (2) No such file or directory 2020-09-12T13:56:14.249 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:14.248+0000 7f5da8b89ec0 -1 bluestore(td/divergent-priors/0/block) _read_bdev_label failed to open td/divergent-priors/0/block: (2) No such file or directory 2020-09-12T13:56:14.249 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:14.248+0000 7f5da8b89ec0 -1 bluestore(td/divergent-priors/0/block) _read_bdev_label failed to open td/divergent-priors/0/block: (2) No such file or directory 2020-09-12T13:56:14.284 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:14.283+0000 7f5da8b89ec0 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2020-09-12T13:56:18.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/divergent-priors/0/keyring 2020-09-12T13:56:18.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T13:56:18.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd0 key to auth repository 2020-09-12T13:56:18.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T13:56:18.485 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd0 key to auth repository 2020-09-12T13:56:19.032 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.0 2020-09-12T13:56:19.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T13:56:19.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 0 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:56:19.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T13:56:19.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T13:56:19.033 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.0 2020-09-12T13:56:19.061 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:19.059+0000 7ff8fd243ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:56:19.083 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:19.081+0000 7ff8fd243ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:56:19.085 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:19.083+0000 7ff8fd243ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:56:19.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 0 2020-09-12T13:56:19.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T13:56:19.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T13:56:19.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T13:56:19.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T13:56:19.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:56:19.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T13:56:19.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:56:19.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T13:56:19.441 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T13:56:19.600 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:19.598+0000 7ff8fd243ec0 -1 Falling back to public interface 2020-09-12T13:56:19.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T13:56:20.410 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:20.408+0000 7ff8fd243ec0 -1 osd.0 0 log_to_monitors {default=true} 2020-09-12T13:56:20.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T13:56:20.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:56:20.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T13:56:20.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:56:20.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T13:56:20.881 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T13:56:21.303 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/21680,v1:127.0.0.1:6803/21680] [v2:127.0.0.1:6804/21680,v1:127.0.0.1:6805/21680] exists,up 97c8d704-849c-44e1-a64d-c66c37f051bf 2020-09-12T13:56:21.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T13:56:21.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T13:56:21.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T13:56:21.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:67: TEST_divergent: for i in $osds 2020-09-12T13:56:21.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:69: TEST_divergent: run_osd td/divergent-priors 1 2020-09-12T13:56:21.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/divergent-priors 2020-09-12T13:56:21.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T13:56:21.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=1 2020-09-12T13:56:21.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T13:56:21.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/divergent-priors/1 2020-09-12T13:56:21.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T13:56:21.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T13:56:21.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T13:56:21.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T13:56:21.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2020-09-12T13:56:21.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2020-09-12T13:56:21.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T13:56:21.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T13:56:21.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T13:56:21.306 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T13:56:21.306 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T13:56:21.306 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T13:56:21.307 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T13:56:21.307 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:56:21.307 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:56:21.307 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:56:21.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:56:21.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T13:56:21.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T13:56:21.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T13:56:21.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T13:56:21.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T13:56:21.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T13:56:21.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T13:56:21.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:56:21.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T13:56:21.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T13:56:21.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/divergent-priors/1 2020-09-12T13:56:21.310 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T13:56:21.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=96117818-e665-481f-9e92-b70e5d147a31 2020-09-12T13:56:21.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd1 96117818-e665-481f-9e92-b70e5d147a31' 2020-09-12T13:56:21.310 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T13:56:21.310 INFO:tasks.workunit.client.0.smithi027.stdout:add osd1 96117818-e665-481f-9e92-b70e5d147a31 2020-09-12T13:56:21.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQCF01xf9bQ9ExAA+RojInEPaVUL72Z35psBow== 2020-09-12T13:56:21.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQCF01xf9bQ9ExAA+RojInEPaVUL72Z35psBow=="}' 2020-09-12T13:56:21.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 96117818-e665-481f-9e92-b70e5d147a31 -i td/divergent-priors/1/new.json 2020-09-12T13:56:21.833 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T13:56:21.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/divergent-priors/1/new.json 2020-09-12T13:56:21.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 1 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQCF01xf9bQ9ExAA+RojInEPaVUL72Z35psBow== --osd-uuid 96117818-e665-481f-9e92-b70e5d147a31 2020-09-12T13:56:21.875 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:21.873+0000 7f7ddece1ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:56:21.877 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:21.875+0000 7f7ddece1ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:56:21.878 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:21.877+0000 7f7ddece1ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:56:21.879 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:21.877+0000 7f7ddece1ec0 -1 bluestore(td/divergent-priors/1/block) _read_bdev_label failed to open td/divergent-priors/1/block: (2) No such file or directory 2020-09-12T13:56:21.879 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:21.877+0000 7f7ddece1ec0 -1 bluestore(td/divergent-priors/1/block) _read_bdev_label failed to open td/divergent-priors/1/block: (2) No such file or directory 2020-09-12T13:56:21.879 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:21.877+0000 7f7ddece1ec0 -1 bluestore(td/divergent-priors/1/block) _read_bdev_label failed to open td/divergent-priors/1/block: (2) No such file or directory 2020-09-12T13:56:21.916 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:21.914+0000 7f7ddece1ec0 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2020-09-12T13:56:26.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/divergent-priors/1/keyring 2020-09-12T13:56:26.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T13:56:26.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd1 key to auth repository 2020-09-12T13:56:26.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T13:56:26.257 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd1 key to auth repository 2020-09-12T13:56:26.811 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T13:56:26.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.1 2020-09-12T13:56:26.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 1 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:56:26.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T13:56:26.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T13:56:26.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T13:56:26.840 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:26.838+0000 7effbb3b0ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:56:26.841 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:26.840+0000 7effbb3b0ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:56:26.843 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:26.841+0000 7effbb3b0ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:56:27.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 1 2020-09-12T13:56:27.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T13:56:27.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T13:56:27.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T13:56:27.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T13:56:27.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:56:27.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T13:56:27.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:56:27.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T13:56:27.221 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T13:56:27.357 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:27.355+0000 7effbb3b0ec0 -1 Falling back to public interface 2020-09-12T13:56:27.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T13:56:28.147 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:28.146+0000 7effbb3b0ec0 -1 osd.1 0 log_to_monitors {default=true} 2020-09-12T13:56:28.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T13:56:28.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:56:28.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T13:56:28.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:56:28.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T13:56:28.638 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T13:56:29.073 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/22674,v1:127.0.0.1:6811/22674] [v2:127.0.0.1:6812/22674,v1:127.0.0.1:6813/22674] exists,up 96117818-e665-481f-9e92-b70e5d147a31 2020-09-12T13:56:29.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T13:56:29.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T13:56:29.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T13:56:29.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:67: TEST_divergent: for i in $osds 2020-09-12T13:56:29.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:69: TEST_divergent: run_osd td/divergent-priors 2 2020-09-12T13:56:29.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/divergent-priors 2020-09-12T13:56:29.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T13:56:29.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=2 2020-09-12T13:56:29.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T13:56:29.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/divergent-priors/2 2020-09-12T13:56:29.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T13:56:29.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T13:56:29.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T13:56:29.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T13:56:29.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2020-09-12T13:56:29.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2020-09-12T13:56:29.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T13:56:29.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T13:56:29.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T13:56:29.076 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T13:56:29.077 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T13:56:29.077 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T13:56:29.079 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T13:56:29.079 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:56:29.079 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:56:29.079 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:56:29.079 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:56:29.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T13:56:29.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T13:56:29.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T13:56:29.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T13:56:29.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T13:56:29.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T13:56:29.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T13:56:29.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:56:29.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T13:56:29.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T13:56:29.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/divergent-priors/2 2020-09-12T13:56:29.082 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T13:56:29.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=cadcbbd5-8cd1-4081-9e23-e263c65059d2 2020-09-12T13:56:29.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd2 cadcbbd5-8cd1-4081-9e23-e263c65059d2' 2020-09-12T13:56:29.082 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T13:56:29.082 INFO:tasks.workunit.client.0.smithi027.stdout:add osd2 cadcbbd5-8cd1-4081-9e23-e263c65059d2 2020-09-12T13:56:29.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQCN01xf2g63BRAAFYZK6b1eN8pCdD+6qp9kuQ== 2020-09-12T13:56:29.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQCN01xf2g63BRAAFYZK6b1eN8pCdD+6qp9kuQ=="}' 2020-09-12T13:56:29.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new cadcbbd5-8cd1-4081-9e23-e263c65059d2 -i td/divergent-priors/2/new.json 2020-09-12T13:56:29.619 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T13:56:29.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/divergent-priors/2/new.json 2020-09-12T13:56:29.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 2 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQCN01xf2g63BRAAFYZK6b1eN8pCdD+6qp9kuQ== --osd-uuid cadcbbd5-8cd1-4081-9e23-e263c65059d2 2020-09-12T13:56:29.662 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:29.660+0000 7f46bad11ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:56:29.664 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:29.662+0000 7f46bad11ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:56:29.665 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:29.664+0000 7f46bad11ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:56:29.666 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:29.664+0000 7f46bad11ec0 -1 bluestore(td/divergent-priors/2/block) _read_bdev_label failed to open td/divergent-priors/2/block: (2) No such file or directory 2020-09-12T13:56:29.666 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:29.664+0000 7f46bad11ec0 -1 bluestore(td/divergent-priors/2/block) _read_bdev_label failed to open td/divergent-priors/2/block: (2) No such file or directory 2020-09-12T13:56:29.666 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:29.664+0000 7f46bad11ec0 -1 bluestore(td/divergent-priors/2/block) _read_bdev_label failed to open td/divergent-priors/2/block: (2) No such file or directory 2020-09-12T13:56:29.747 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:29.746+0000 7f46bad11ec0 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2020-09-12T13:56:33.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/divergent-priors/2/keyring 2020-09-12T13:56:33.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T13:56:33.782 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd2 key to auth repository 2020-09-12T13:56:33.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd2 key to auth repository 2020-09-12T13:56:33.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T13:56:34.332 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.2 2020-09-12T13:56:34.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.2 2020-09-12T13:56:34.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 2 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:56:34.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T13:56:34.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T13:56:34.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T13:56:34.361 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:34.360+0000 7f14a5caeec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:56:34.362 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:34.362+0000 7f14a5caeec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:56:34.363 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:34.363+0000 7f14a5caeec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:56:34.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 2 2020-09-12T13:56:34.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T13:56:34.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T13:56:34.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T13:56:34.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T13:56:34.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:56:34.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T13:56:34.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:56:34.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T13:56:34.742 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T13:56:34.882 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:34.881+0000 7f14a5caeec0 -1 Falling back to public interface 2020-09-12T13:56:35.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T13:56:35.739 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:56:35.738+0000 7f14a5caeec0 -1 osd.2 0 log_to_monitors {default=true} 2020-09-12T13:56:36.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T13:56:36.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:56:36.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T13:56:36.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:56:36.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T13:56:36.161 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T13:56:36.586 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/23668,v1:127.0.0.1:6819/23668] [v2:127.0.0.1:6820/23668,v1:127.0.0.1:6821/23668] exists,up cadcbbd5-8cd1-4081-9e23-e263c65059d2 2020-09-12T13:56:36.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T13:56:36.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T13:56:36.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T13:56:36.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:72: TEST_divergent: ceph osd set noout 2020-09-12T13:56:37.416 INFO:tasks.workunit.client.0.smithi027.stderr:noout is set 2020-09-12T13:56:37.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:73: TEST_divergent: ceph osd set noin 2020-09-12T13:56:38.261 INFO:tasks.workunit.client.0.smithi027.stderr:noin is set 2020-09-12T13:56:38.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:74: TEST_divergent: ceph osd set nodown 2020-09-12T13:56:38.997 INFO:tasks.workunit.client.0.smithi027.stderr:nodown is set 2020-09-12T13:56:39.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:75: TEST_divergent: create_pool test 1 1 2020-09-12T13:56:39.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test 1 1 2020-09-12T13:56:39.800 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test' already exists 2020-09-12T13:56:39.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T13:56:40.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:76: TEST_divergent: ceph osd pool set test size 3 2020-09-12T13:56:41.781 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 1 size to 3 2020-09-12T13:56:41.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:77: TEST_divergent: ceph osd pool set test min_size 2 2020-09-12T13:56:42.448 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 1 min_size to 2 2020-09-12T13:56:42.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:79: TEST_divergent: flush_pg_stats 2020-09-12T13:56:42.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T13:56:42.458 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T13:56:42.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T13:56:42.873 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T13:56:42.874 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T13:56:42.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T13:56:42.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T13:56:42.874 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T13:56:43.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836487 2020-09-12T13:56:43.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836487 2020-09-12T13:56:43.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836487' 2020-09-12T13:56:43.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T13:56:43.023 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T13:56:43.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672965 2020-09-12T13:56:43.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672965 2020-09-12T13:56:43.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836487 1-42949672965' 2020-09-12T13:56:43.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T13:56:43.170 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T13:56:43.316 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509445 2020-09-12T13:56:43.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509445 2020-09-12T13:56:43.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836487 1-42949672965 2-64424509445' 2020-09-12T13:56:43.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T13:56:43.318 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836487 2020-09-12T13:56:43.318 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T13:56:43.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T13:56:43.319 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836487 2020-09-12T13:56:43.319 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T13:56:43.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836487 2020-09-12T13:56:43.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2020-09-12T13:56:43.321 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836487 2020-09-12T13:56:43.321 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T13:56:43.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836487 -lt 21474836487 2020-09-12T13:56:43.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T13:56:43.745 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672965 2020-09-12T13:56:43.745 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T13:56:43.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T13:56:43.747 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672965 2020-09-12T13:56:43.747 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T13:56:43.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672965 2020-09-12T13:56:43.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672965' 2020-09-12T13:56:43.748 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672965 2020-09-12T13:56:43.749 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T13:56:44.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672965 -lt 42949672965 2020-09-12T13:56:44.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T13:56:44.167 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509445 2020-09-12T13:56:44.167 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T13:56:44.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T13:56:44.169 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509445 2020-09-12T13:56:44.169 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T13:56:44.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509445 2020-09-12T13:56:44.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509445' 2020-09-12T13:56:44.170 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509445 2020-09-12T13:56:44.171 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T13:56:44.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509445 -lt 64424509445 2020-09-12T13:56:44.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:80: TEST_divergent: wait_for_clean 2020-09-12T13:56:44.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T13:56:44.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T13:56:44.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T13:56:44.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T13:56:44.581 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 90 .1 2020-09-12T13:56:44.581 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T13:56:44.581 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T13:56:44.581 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T13:56:44.581 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T13:56:44.581 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T13:56:44.673 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T13:56:44.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T13:56:44.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T13:56:44.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T13:56:44.675 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T13:56:45.085 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T13:56:45.085 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T13:56:45.085 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T13:56:45.085 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T13:56:45.086 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T13:56:45.086 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T13:56:45.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836489 2020-09-12T13:56:45.236 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836489 2020-09-12T13:56:45.236 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836489' 2020-09-12T13:56:45.236 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T13:56:45.237 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T13:56:45.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672968 2020-09-12T13:56:45.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672968 2020-09-12T13:56:45.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836489 1-42949672968' 2020-09-12T13:56:45.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T13:56:45.385 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T13:56:45.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509447 2020-09-12T13:56:45.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509447 2020-09-12T13:56:45.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836489 1-42949672968 2-64424509447' 2020-09-12T13:56:45.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T13:56:45.532 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836489 2020-09-12T13:56:45.532 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T13:56:45.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T13:56:45.533 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836489 2020-09-12T13:56:45.534 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T13:56:45.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836489 2020-09-12T13:56:45.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2020-09-12T13:56:45.535 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836489 2020-09-12T13:56:45.535 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T13:56:45.944 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836489 -lt 21474836489 2020-09-12T13:56:45.944 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T13:56:45.945 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672968 2020-09-12T13:56:45.945 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T13:56:45.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T13:56:45.947 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672968 2020-09-12T13:56:45.947 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T13:56:45.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672968 2020-09-12T13:56:45.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672968' 2020-09-12T13:56:45.948 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672968 2020-09-12T13:56:45.949 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T13:56:46.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672968 -lt 42949672968 2020-09-12T13:56:46.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T13:56:46.386 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509447 2020-09-12T13:56:46.386 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T13:56:46.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T13:56:46.388 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509447 2020-09-12T13:56:46.388 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T13:56:46.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509447 2020-09-12T13:56:46.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509447' 2020-09-12T13:56:46.389 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509447 2020-09-12T13:56:46.390 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T13:56:46.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509446 -lt 64424509447 2020-09-12T13:56:46.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T13:56:47.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T13:56:47.802 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T13:56:48.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509448 -lt 64424509447 2020-09-12T13:56:48.235 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T13:56:48.236 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T13:56:48.236 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T13:56:48.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 1 == 0 2020-09-12T13:56:49.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T13:56:49.300 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T13:56:49.300 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T13:56:49.301 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T13:56:49.301 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T13:56:49.301 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T13:56:49.301 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T13:56:49.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T13:56:49.301 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T13:56:49.301 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T13:56:49.302 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T13:56:49.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 1 2020-09-12T13:56:49.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T13:56:49.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T13:56:49.714 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:83: TEST_divergent: ceph pg dump pgs --format=json 2020-09-12T13:56:49.714 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:83: TEST_divergent: jq '.pg_stats[0].up_primary' 2020-09-12T13:56:50.105 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T13:56:50.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:83: TEST_divergent: local divergent=1 2020-09-12T13:56:50.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:84: TEST_divergent: echo 'primary and soon to be divergent is 1' 2020-09-12T13:56:50.116 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:85: TEST_divergent: ceph pg dump pgs 2020-09-12T13:56:50.116 INFO:tasks.workunit.client.0.smithi027.stdout:primary and soon to be divergent is 1 2020-09-12T13:56:50.501 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T13:56:50.502 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 0 0 0 0 0 0 0 0 0 0 active+clean 2020-09-12T13:56:40.891087+0000 0'0 28:11 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T13:56:39.394566+0000 0'0 2020-09-12T13:56:39.394566+0000 0 2020-09-12T13:56:50.502 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T13:56:50.502 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T13:56:50.503 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T13:56:50.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:86: TEST_divergent: local non_divergent= 2020-09-12T13:56:50.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:87: TEST_divergent: for i in $osds 2020-09-12T13:56:50.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:89: TEST_divergent: '[' 0 = 1 ']' 2020-09-12T13:56:50.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:92: TEST_divergent: non_divergent=' 0' 2020-09-12T13:56:50.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:87: TEST_divergent: for i in $osds 2020-09-12T13:56:50.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:89: TEST_divergent: '[' 1 = 1 ']' 2020-09-12T13:56:50.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:90: TEST_divergent: continue 2020-09-12T13:56:50.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:87: TEST_divergent: for i in $osds 2020-09-12T13:56:50.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:89: TEST_divergent: '[' 2 = 1 ']' 2020-09-12T13:56:50.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:92: TEST_divergent: non_divergent=' 0 2' 2020-09-12T13:56:50.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:95: TEST_divergent: echo 'writing initial objects' 2020-09-12T13:56:50.515 INFO:tasks.workunit.client.0.smithi027.stdout:writing initial objects 2020-09-12T13:56:50.515 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: seq 1 100 2020-09-12T13:56:50.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:50.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_1 /etc/fstab 2020-09-12T13:56:50.673 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:50.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_2 /etc/fstab 2020-09-12T13:56:50.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:50.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_3 /etc/fstab 2020-09-12T13:56:50.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:50.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_4 /etc/fstab 2020-09-12T13:56:51.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:51.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_5 /etc/fstab 2020-09-12T13:56:51.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:51.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_6 /etc/fstab 2020-09-12T13:56:51.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:51.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_7 /etc/fstab 2020-09-12T13:56:51.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:51.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_8 /etc/fstab 2020-09-12T13:56:51.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:51.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_9 /etc/fstab 2020-09-12T13:56:51.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:51.864 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_10 /etc/fstab 2020-09-12T13:56:52.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:52.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_11 /etc/fstab 2020-09-12T13:56:52.165 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:52.165 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_12 /etc/fstab 2020-09-12T13:56:52.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:52.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_13 /etc/fstab 2020-09-12T13:56:52.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:52.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_14 /etc/fstab 2020-09-12T13:56:52.584 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:52.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_15 /etc/fstab 2020-09-12T13:56:52.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:52.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_16 /etc/fstab 2020-09-12T13:56:52.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:52.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_17 /etc/fstab 2020-09-12T13:56:53.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:53.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_18 /etc/fstab 2020-09-12T13:56:53.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:53.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_19 /etc/fstab 2020-09-12T13:56:53.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:53.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_20 /etc/fstab 2020-09-12T13:56:53.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:53.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_21 /etc/fstab 2020-09-12T13:56:53.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:53.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_22 /etc/fstab 2020-09-12T13:56:53.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:53.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_23 /etc/fstab 2020-09-12T13:56:53.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:53.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_24 /etc/fstab 2020-09-12T13:56:54.009 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:54.009 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_25 /etc/fstab 2020-09-12T13:56:54.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:54.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_26 /etc/fstab 2020-09-12T13:56:54.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:54.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_27 /etc/fstab 2020-09-12T13:56:54.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:54.437 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_28 /etc/fstab 2020-09-12T13:56:54.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:54.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_29 /etc/fstab 2020-09-12T13:56:54.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:54.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_30 /etc/fstab 2020-09-12T13:56:54.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:54.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_31 /etc/fstab 2020-09-12T13:56:54.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:54.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_32 /etc/fstab 2020-09-12T13:56:55.106 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:55.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_33 /etc/fstab 2020-09-12T13:56:55.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:55.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_34 /etc/fstab 2020-09-12T13:56:55.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:55.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_35 /etc/fstab 2020-09-12T13:56:55.551 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:55.551 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_36 /etc/fstab 2020-09-12T13:56:55.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:55.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_37 /etc/fstab 2020-09-12T13:56:55.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:55.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_38 /etc/fstab 2020-09-12T13:56:55.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:55.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_39 /etc/fstab 2020-09-12T13:56:56.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:56.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_40 /etc/fstab 2020-09-12T13:56:56.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:56.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_41 /etc/fstab 2020-09-12T13:56:56.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:56.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_42 /etc/fstab 2020-09-12T13:56:56.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:56.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_43 /etc/fstab 2020-09-12T13:56:56.666 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:56.666 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_44 /etc/fstab 2020-09-12T13:56:56.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:56.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_45 /etc/fstab 2020-09-12T13:56:56.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:56.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_46 /etc/fstab 2020-09-12T13:56:57.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:57.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_47 /etc/fstab 2020-09-12T13:56:57.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:57.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_48 /etc/fstab 2020-09-12T13:56:57.371 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:57.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_49 /etc/fstab 2020-09-12T13:56:57.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:57.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_50 /etc/fstab 2020-09-12T13:56:57.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:57.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_51 /etc/fstab 2020-09-12T13:56:57.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:57.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_52 /etc/fstab 2020-09-12T13:56:57.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:57.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_53 /etc/fstab 2020-09-12T13:56:58.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:58.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_54 /etc/fstab 2020-09-12T13:56:58.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:58.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_55 /etc/fstab 2020-09-12T13:56:58.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:58.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_56 /etc/fstab 2020-09-12T13:56:58.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:58.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_57 /etc/fstab 2020-09-12T13:56:58.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:58.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_58 /etc/fstab 2020-09-12T13:56:58.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:58.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_59 /etc/fstab 2020-09-12T13:56:58.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:58.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_60 /etc/fstab 2020-09-12T13:56:59.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:59.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_61 /etc/fstab 2020-09-12T13:56:59.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:59.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_62 /etc/fstab 2020-09-12T13:56:59.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:59.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_63 /etc/fstab 2020-09-12T13:56:59.550 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:59.550 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_64 /etc/fstab 2020-09-12T13:56:59.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:59.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_65 /etc/fstab 2020-09-12T13:56:59.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:59.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_66 /etc/fstab 2020-09-12T13:56:59.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:56:59.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_67 /etc/fstab 2020-09-12T13:57:00.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:00.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_68 /etc/fstab 2020-09-12T13:57:00.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:00.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_69 /etc/fstab 2020-09-12T13:57:00.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:00.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_70 /etc/fstab 2020-09-12T13:57:00.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:00.573 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_71 /etc/fstab 2020-09-12T13:57:00.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:00.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_72 /etc/fstab 2020-09-12T13:57:00.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:00.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_73 /etc/fstab 2020-09-12T13:57:00.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:01.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_74 /etc/fstab 2020-09-12T13:57:01.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:01.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_75 /etc/fstab 2020-09-12T13:57:01.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:01.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_76 /etc/fstab 2020-09-12T13:57:01.418 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:01.418 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_77 /etc/fstab 2020-09-12T13:57:01.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:01.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_78 /etc/fstab 2020-09-12T13:57:01.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:01.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_79 /etc/fstab 2020-09-12T13:57:01.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:01.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_80 /etc/fstab 2020-09-12T13:57:01.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:01.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_81 /etc/fstab 2020-09-12T13:57:02.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:02.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_82 /etc/fstab 2020-09-12T13:57:02.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:02.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_83 /etc/fstab 2020-09-12T13:57:02.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:02.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_84 /etc/fstab 2020-09-12T13:57:02.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:02.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_85 /etc/fstab 2020-09-12T13:57:02.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:02.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_86 /etc/fstab 2020-09-12T13:57:02.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:02.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_87 /etc/fstab 2020-09-12T13:57:03.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:03.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_88 /etc/fstab 2020-09-12T13:57:03.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:03.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_89 /etc/fstab 2020-09-12T13:57:03.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:03.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_90 /etc/fstab 2020-09-12T13:57:03.438 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:03.438 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_91 /etc/fstab 2020-09-12T13:57:03.598 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:03.598 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_92 /etc/fstab 2020-09-12T13:57:03.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:03.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_93 /etc/fstab 2020-09-12T13:57:03.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:03.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_94 /etc/fstab 2020-09-12T13:57:03.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:04.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_95 /etc/fstab 2020-09-12T13:57:04.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:04.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_96 /etc/fstab 2020-09-12T13:57:04.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:04.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_97 /etc/fstab 2020-09-12T13:57:04.468 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:04.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_98 /etc/fstab 2020-09-12T13:57:04.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:04.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_99 /etc/fstab 2020-09-12T13:57:04.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2020-09-12T13:57:04.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_100 /etc/fstab 2020-09-12T13:57:04.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:102: TEST_divergent: WAIT_FOR_CLEAN_TIMEOUT=20 2020-09-12T13:57:04.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:102: TEST_divergent: wait_for_clean 2020-09-12T13:57:04.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T13:57:04.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T13:57:04.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T13:57:04.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T13:57:04.856 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 20 .1 2020-09-12T13:57:04.856 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T13:57:04.856 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T13:57:04.856 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T13:57:04.857 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T13:57:04.857 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T13:57:04.915 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T13:57:04.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T13:57:04.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T13:57:04.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T13:57:04.916 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T13:57:05.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T13:57:05.331 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T13:57:05.331 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T13:57:05.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T13:57:05.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T13:57:05.332 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T13:57:05.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836495 2020-09-12T13:57:05.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836495 2020-09-12T13:57:05.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836495' 2020-09-12T13:57:05.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T13:57:05.480 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T13:57:05.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672974 2020-09-12T13:57:05.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672974 2020-09-12T13:57:05.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836495 1-42949672974' 2020-09-12T13:57:05.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T13:57:05.628 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T13:57:05.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509453 2020-09-12T13:57:05.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509453 2020-09-12T13:57:05.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836495 1-42949672974 2-64424509453' 2020-09-12T13:57:05.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T13:57:05.776 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836495 2020-09-12T13:57:05.776 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T13:57:05.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T13:57:05.777 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836495 2020-09-12T13:57:05.777 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T13:57:05.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836495 2020-09-12T13:57:05.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836495' 2020-09-12T13:57:05.779 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836495 2020-09-12T13:57:05.779 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T13:57:06.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836495 -lt 21474836495 2020-09-12T13:57:06.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T13:57:06.207 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672974 2020-09-12T13:57:06.207 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T13:57:06.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T13:57:06.209 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672974 2020-09-12T13:57:06.209 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T13:57:06.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672974 2020-09-12T13:57:06.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672974' 2020-09-12T13:57:06.210 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672974 2020-09-12T13:57:06.211 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T13:57:06.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672972 -lt 42949672974 2020-09-12T13:57:06.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T13:57:07.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T13:57:07.649 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T13:57:08.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672974 -lt 42949672974 2020-09-12T13:57:08.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T13:57:08.069 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509453 2020-09-12T13:57:08.069 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T13:57:08.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T13:57:08.071 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509453 2020-09-12T13:57:08.071 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T13:57:08.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509453 2020-09-12T13:57:08.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509453' 2020-09-12T13:57:08.073 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509453 2020-09-12T13:57:08.073 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T13:57:08.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509454 -lt 64424509453 2020-09-12T13:57:08.480 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T13:57:08.480 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T13:57:08.481 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T13:57:09.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 1 == 0 2020-09-12T13:57:09.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T13:57:09.014 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T13:57:09.014 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T13:57:09.015 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T13:57:09.015 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T13:57:09.015 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T13:57:09.015 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T13:57:09.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T13:57:09.408 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T13:57:09.408 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T13:57:09.409 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T13:57:09.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 1 2020-09-12T13:57:09.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T13:57:09.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T13:57:09.957 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:104: TEST_divergent: get_pg test existing_1 2020-09-12T13:57:09.957 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1031: get_pg: local poolname=test 2020-09-12T13:57:09.957 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1032: get_pg: local objectname=existing_1 2020-09-12T13:57:09.957 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1034: get_pg: ceph --format json osd map test existing_1 2020-09-12T13:57:09.957 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1034: get_pg: jq -r .pgid 2020-09-12T13:57:10.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:104: TEST_divergent: local pgid=1.0 2020-09-12T13:57:10.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:107: TEST_divergent: echo 'blackholing osds 0 2' 2020-09-12T13:57:10.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:108: TEST_divergent: ceph pg dump pgs 2020-09-12T13:57:10.370 INFO:tasks.workunit.client.0.smithi027.stdout:blackholing osds 0 2 2020-09-12T13:57:10.766 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T13:57:10.766 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 0 0 0 0 48300 0 0 16 16 active+clean 2020-09-12T13:56:40.891087+0000 29'100 29:111 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T13:56:39.394566+0000 0'0 2020-09-12T13:56:39.394566+0000 0 2020-09-12T13:57:10.766 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T13:57:10.766 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T13:57:10.767 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T13:57:10.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:109: TEST_divergent: for i in $non_divergent 2020-09-12T13:57:10.778 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: get_asok_path osd.0 2020-09-12T13:57:10.778 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2020-09-12T13:57:10.778 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2020-09-12T13:57:10.778 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T13:57:10.779 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:57:10.779 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:57:10.779 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.0.asok 2020-09-12T13:57:10.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: CEPH_ARGS= 2020-09-12T13:57:10.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.0.asok config set objectstore_blackhole 1 2020-09-12T13:57:10.880 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T13:57:10.880 INFO:tasks.workunit.client.0.smithi027.stdout: "success": "objectstore_blackhole = 'true' (not observed, change may require restart) " 2020-09-12T13:57:10.881 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T13:57:10.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:109: TEST_divergent: for i in $non_divergent 2020-09-12T13:57:10.889 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: get_asok_path osd.2 2020-09-12T13:57:10.889 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2020-09-12T13:57:10.890 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2020-09-12T13:57:10.890 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T13:57:10.890 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:57:10.890 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:57:10.891 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.2.asok 2020-09-12T13:57:10.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: CEPH_ARGS= 2020-09-12T13:57:10.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.2.asok config set objectstore_blackhole 1 2020-09-12T13:57:10.991 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T13:57:10.992 INFO:tasks.workunit.client.0.smithi027.stdout: "success": "objectstore_blackhole = 'true' (not observed, change may require restart) " 2020-09-12T13:57:10.992 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T13:57:10.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:114: TEST_divergent: local case5=100 2020-09-12T13:57:11.000 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:115: TEST_divergent: expr 100 - 1 2020-09-12T13:57:11.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:115: TEST_divergent: local case3=99 2020-09-12T13:57:11.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:117: TEST_divergent: echo 'writing divergent object' 2020-09-12T13:57:11.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:119: TEST_divergent: echo 'create missing divergent object' 2020-09-12T13:57:11.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:120: TEST_divergent: inject_eio rep data test existing_99 td/divergent-priors 0 2020-09-12T13:57:11.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:118: TEST_divergent: rados -p test put existing_100 /etc/fstab 2020-09-12T13:57:11.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2231: inject_eio: local pooltype=rep 2020-09-12T13:57:11.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2232: inject_eio: shift 2020-09-12T13:57:11.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2233: inject_eio: local which=data 2020-09-12T13:57:11.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2234: inject_eio: shift 2020-09-12T13:57:11.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2235: inject_eio: local poolname=test 2020-09-12T13:57:11.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2236: inject_eio: shift 2020-09-12T13:57:11.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: inject_eio: local objname=existing_99 2020-09-12T13:57:11.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2238: inject_eio: shift 2020-09-12T13:57:11.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: inject_eio: local dir=td/divergent-priors 2020-09-12T13:57:11.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2240: inject_eio: shift 2020-09-12T13:57:11.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2241: inject_eio: local shard_id=0 2020-09-12T13:57:11.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2242: inject_eio: shift 2020-09-12T13:57:11.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2244: inject_eio: initial_osds=($(get_osds $poolname $objname)) 2020-09-12T13:57:11.005 INFO:tasks.workunit.client.0.smithi027.stdout:writing divergent object 2020-09-12T13:57:11.005 INFO:tasks.workunit.client.0.smithi027.stdout:create missing divergent object 2020-09-12T13:57:11.005 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2244: inject_eio: get_osds test existing_99 2020-09-12T13:57:11.006 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:959: get_osds: local poolname=test 2020-09-12T13:57:11.006 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:960: get_osds: local objectname=existing_99 2020-09-12T13:57:11.006 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:963: get_osds: ceph --format json osd map test existing_99 2020-09-12T13:57:11.006 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:963: get_osds: jq '.acting | .[]' 2020-09-12T13:57:11.416 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:963: get_osds: local 'osds=1 2020-09-12T13:57:11.416 INFO:tasks.workunit.client.0.smithi027.stderr:0 2020-09-12T13:57:11.416 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T13:57:11.417 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:965: get_osds: echo 1 0 2 2020-09-12T13:57:11.417 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2244: inject_eio: local -a initial_osds 2020-09-12T13:57:11.417 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2245: inject_eio: local osd_id=1 2020-09-12T13:57:11.417 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2246: inject_eio: '[' rep '!=' ec ']' 2020-09-12T13:57:11.417 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2247: inject_eio: shard_id= 2020-09-12T13:57:11.418 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2249: inject_eio: cat td/divergent-priors/1/type 2020-09-12T13:57:11.418 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2249: inject_eio: type=bluestore 2020-09-12T13:57:11.418 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2250: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2020-09-12T13:57:11.418 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1099: set_config: local daemon=osd 2020-09-12T13:57:11.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1100: set_config: local id=1 2020-09-12T13:57:11.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1101: set_config: local config=bluestore_debug_inject_read_err 2020-09-12T13:57:11.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1102: set_config: local value=true 2020-09-12T13:57:11.420 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1106: set_config: jq 'has("success")' 2020-09-12T13:57:11.420 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1106: set_config: get_asok_path osd.1 2020-09-12T13:57:11.420 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2020-09-12T13:57:11.420 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2020-09-12T13:57:11.420 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T13:57:11.420 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:57:11.421 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:57:11.421 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.1.asok 2020-09-12T13:57:11.421 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1106: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.20815/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2020-09-12T13:57:11.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1106: set_config: test true == true 2020-09-12T13:57:11.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2251: inject_eio: local loop=0 2020-09-12T13:57:11.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: inject_eio: grep -q Invalid 2020-09-12T13:57:11.532 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2252: inject_eio: get_asok_path osd.1 2020-09-12T13:57:11.532 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2020-09-12T13:57:11.532 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2020-09-12T13:57:11.532 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T13:57:11.532 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:57:11.532 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:57:11.532 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.1.asok 2020-09-12T13:57:11.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2252: inject_eio: CEPH_ARGS= 2020-09-12T13:57:11.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2252: inject_eio: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.1.asok injectdataerr test existing_99 2020-09-12T13:57:11.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:122: TEST_divergent: sleep 10 2020-09-12T13:57:11.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:121: TEST_divergent: rados -p test get existing_99 td/divergent-priors/existing 2020-09-12T13:57:21.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:123: TEST_divergent: killall -9 rados 2020-09-12T13:57:21.668 INFO:tasks.workunit.client.0.smithi027.stderr:rados: no process found 2020-09-12T13:57:21.669 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:126: TEST_divergent: echo 'killing all the osds' 2020-09-12T13:57:21.669 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:127: TEST_divergent: ceph pg dump pgs 2020-09-12T13:57:21.669 INFO:tasks.workunit.client.0.smithi027.stdout:killing all the osds 2020-09-12T13:57:22.062 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T13:57:22.062 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 0 0 0 0 48300 0 0 17 17 active+clean 2020-09-12T13:57:11.984140+0000 29'101 29:120 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T13:56:39.394566+0000 0'0 2020-09-12T13:56:39.394566+0000 0 2020-09-12T13:57:22.062 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T13:57:22.062 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T13:57:22.064 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T13:57:22.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:128: TEST_divergent: kill_daemons td/divergent-priors KILL osd 2020-09-12T13:57:22.074 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T13:57:22.074 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T13:57:22.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T13:57:22.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T13:57:22.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T13:57:22.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T13:57:22.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:129: TEST_divergent: for i in $osds 2020-09-12T13:57:22.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:131: TEST_divergent: ceph osd down osd.0 2020-09-12T13:57:22.712 INFO:tasks.workunit.client.0.smithi027.stderr:osd.0 is already down. 2020-09-12T13:57:22.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:129: TEST_divergent: for i in $osds 2020-09-12T13:57:22.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:131: TEST_divergent: ceph osd down osd.1 2020-09-12T13:57:23.262 INFO:tasks.workunit.client.0.smithi027.stderr:osd.1 is already down. 2020-09-12T13:57:23.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:129: TEST_divergent: for i in $osds 2020-09-12T13:57:23.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:131: TEST_divergent: ceph osd down osd.2 2020-09-12T13:57:23.863 INFO:tasks.workunit.client.0.smithi027.stderr:osd.2 is already down. 2020-09-12T13:57:23.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:133: TEST_divergent: for i in $non_divergent 2020-09-12T13:57:23.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:135: TEST_divergent: ceph osd out osd.0 2020-09-12T13:57:24.432 INFO:tasks.workunit.client.0.smithi027.stderr:osd.0 is already out. 2020-09-12T13:57:24.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:133: TEST_divergent: for i in $non_divergent 2020-09-12T13:57:24.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:135: TEST_divergent: ceph osd out osd.2 2020-09-12T13:57:25.000 INFO:tasks.workunit.client.0.smithi027.stderr:osd.2 is already out. 2020-09-12T13:57:25.009 INFO:tasks.workunit.client.0.smithi027.stdout:bringing up non_divergent 0 2 2020-09-12T13:57:25.009 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:139: TEST_divergent: echo 'bringing up non_divergent 0 2' 2020-09-12T13:57:25.009 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:140: TEST_divergent: ceph pg dump pgs 2020-09-12T13:57:25.396 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T13:57:25.396 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 0 0 0 0 48300 0 0 17 17 stale+active+clean 2020-09-12T13:57:11.984140+0000 29'101 29:120 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T13:56:39.394566+0000 0'0 2020-09-12T13:56:39.394566+0000 0 2020-09-12T13:57:25.397 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T13:57:25.397 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T13:57:25.398 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T13:57:25.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:141: TEST_divergent: for i in $non_divergent 2020-09-12T13:57:25.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:143: TEST_divergent: activate_osd td/divergent-priors 0 2020-09-12T13:57:25.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/divergent-priors 2020-09-12T13:57:25.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T13:57:25.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=0 2020-09-12T13:57:25.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T13:57:25.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/divergent-priors/0 2020-09-12T13:57:25.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T13:57:25.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T13:57:25.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T13:57:25.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T13:57:25.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2020-09-12T13:57:25.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2020-09-12T13:57:25.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T13:57:25.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T13:57:25.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T13:57:25.412 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T13:57:25.412 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T13:57:25.412 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T13:57:25.412 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T13:57:25.412 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:57:25.412 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:57:25.413 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:57:25.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:57:25.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T13:57:25.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T13:57:25.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T13:57:25.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T13:57:25.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T13:57:25.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:57:25.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T13:57:25.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T13:57:25.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/divergent-priors/0 2020-09-12T13:57:25.415 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.0 2020-09-12T13:57:25.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.0 2020-09-12T13:57:25.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 0 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:57:25.415 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/divergent-priors/0/whoami 2020-09-12T13:57:25.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 0 = 0 ']' 2020-09-12T13:57:25.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T13:57:25.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T13:57:25.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T13:57:25.439 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:57:25.438+0000 7f2ad84f0ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:57:25.440 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:57:25.440+0000 7f2ad84f0ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:57:25.441 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:57:25.441+0000 7f2ad84f0ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:57:25.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: wait_for_osd up 0 2020-09-12T13:57:25.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T13:57:25.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T13:57:25.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T13:57:25.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T13:57:25.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:57:25.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T13:57:25.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:57:25.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T13:57:25.828 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T13:57:25.962 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:57:25.961+0000 7f2ad84f0ec0 -1 Falling back to public interface 2020-09-12T13:57:26.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T13:57:27.124 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:57:27.123+0000 7f2ad84f0ec0 -1 osd.0 29 log_to_monitors {default=true} 2020-09-12T13:57:27.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T13:57:27.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:57:27.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T13:57:27.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:57:27.237 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T13:57:27.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T13:57:27.703 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up out weight 0 up_from 35 up_thru 0 down_at 30 last_clean_interval [5,29) [v2:127.0.0.1:6802/27534,v1:127.0.0.1:6803/27534] [v2:127.0.0.1:6804/27534,v1:127.0.0.1:6805/27534] exists,up 97c8d704-849c-44e1-a64d-c66c37f051bf 2020-09-12T13:57:27.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T13:57:27.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T13:57:27.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T13:57:27.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:141: TEST_divergent: for i in $non_divergent 2020-09-12T13:57:27.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:143: TEST_divergent: activate_osd td/divergent-priors 2 2020-09-12T13:57:27.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/divergent-priors 2020-09-12T13:57:27.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T13:57:27.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=2 2020-09-12T13:57:27.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T13:57:27.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/divergent-priors/2 2020-09-12T13:57:27.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T13:57:27.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T13:57:27.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T13:57:27.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T13:57:27.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2020-09-12T13:57:27.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2020-09-12T13:57:27.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T13:57:27.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T13:57:27.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T13:57:27.706 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T13:57:27.706 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T13:57:27.707 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T13:57:27.707 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T13:57:27.707 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:57:27.707 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:57:27.707 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:57:27.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:57:27.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T13:57:27.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T13:57:27.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T13:57:27.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T13:57:27.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T13:57:27.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:57:27.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T13:57:27.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T13:57:27.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/divergent-priors/2 2020-09-12T13:57:27.709 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.2 2020-09-12T13:57:27.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.2 2020-09-12T13:57:27.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 2 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:57:27.710 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/divergent-priors/2/whoami 2020-09-12T13:57:27.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 2 = 2 ']' 2020-09-12T13:57:27.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T13:57:27.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T13:57:27.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T13:57:27.733 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:57:27.732+0000 7f583183cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:57:27.735 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:57:27.734+0000 7f583183cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:57:27.736 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:57:27.736+0000 7f583183cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:57:28.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: wait_for_osd up 2 2020-09-12T13:57:28.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T13:57:28.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T13:57:28.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T13:57:28.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T13:57:28.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:57:28.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T13:57:28.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:57:28.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T13:57:28.118 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T13:57:28.260 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:57:28.259+0000 7f583183cec0 -1 Falling back to public interface 2020-09-12T13:57:28.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T13:57:29.506 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:57:29.505+0000 7f583183cec0 -1 osd.2 29 log_to_monitors {default=true} 2020-09-12T13:57:29.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T13:57:29.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:57:29.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T13:57:29.542 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T13:57:29.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T13:57:29.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:57:29.963 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up out weight 0 up_from 36 up_thru 0 down_at 32 last_clean_interval [15,29) [v2:127.0.0.1:6810/27933,v1:127.0.0.1:6811/27933] [v2:127.0.0.1:6812/27933,v1:127.0.0.1:6813/27933] exists,up cadcbbd5-8cd1-4081-9e23-e263c65059d2 2020-09-12T13:57:29.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T13:57:29.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T13:57:29.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T13:57:29.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:145: TEST_divergent: for i in $non_divergent 2020-09-12T13:57:29.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:147: TEST_divergent: ceph osd in osd.0 2020-09-12T13:57:30.553 INFO:tasks.workunit.client.0.smithi027.stderr:osd.0 is already in. 2020-09-12T13:57:30.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:145: TEST_divergent: for i in $non_divergent 2020-09-12T13:57:30.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:147: TEST_divergent: ceph osd in osd.2 2020-09-12T13:57:31.338 INFO:tasks.workunit.client.0.smithi027.stderr:osd.2 is already in. 2020-09-12T13:57:31.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:150: TEST_divergent: WAIT_FOR_CLEAN_TIMEOUT=20 2020-09-12T13:57:31.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:150: TEST_divergent: wait_for_clean 2020-09-12T13:57:31.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T13:57:31.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T13:57:31.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T13:57:31.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T13:57:31.349 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 20 .1 2020-09-12T13:57:31.349 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T13:57:31.350 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T13:57:31.350 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T13:57:31.350 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T13:57:31.350 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T13:57:31.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T13:57:31.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T13:57:31.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T13:57:31.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T13:57:31.407 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T13:57:31.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T13:57:31.818 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T13:57:31.818 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T13:57:31.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T13:57:31.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T13:57:31.819 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T13:57:31.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=150323855363 2020-09-12T13:57:31.966 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 150323855363 2020-09-12T13:57:31.966 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-150323855363' 2020-09-12T13:57:31.966 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T13:57:31.966 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T13:57:32.072 INFO:tasks.workunit.client.0.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-09-12T13:57:32.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq= 2020-09-12T13:57:32.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z '' 2020-09-12T13:57:32.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2075: flush_pg_stats: continue 2020-09-12T13:57:32.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T13:57:32.074 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T13:57:32.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=154618822659 2020-09-12T13:57:32.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 154618822659 2020-09-12T13:57:32.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-150323855363 2-154618822659' 2020-09-12T13:57:32.222 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T13:57:32.222 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-150323855363 2020-09-12T13:57:32.223 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T13:57:32.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T13:57:32.223 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-150323855363 2020-09-12T13:57:32.223 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T13:57:32.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=150323855363 2020-09-12T13:57:32.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 150323855363' 2020-09-12T13:57:32.224 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T13:57:32.225 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 150323855363 2020-09-12T13:57:32.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 1 -lt 150323855363 2020-09-12T13:57:32.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T13:57:33.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T13:57:33.630 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T13:57:34.085 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 150323855364 -lt 150323855363 2020-09-12T13:57:34.086 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T13:57:34.086 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-154618822659 2020-09-12T13:57:34.086 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T13:57:34.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T13:57:34.087 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-154618822659 2020-09-12T13:57:34.087 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T13:57:34.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=154618822659 2020-09-12T13:57:34.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 154618822659' 2020-09-12T13:57:34.089 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 154618822659 2020-09-12T13:57:34.089 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T13:57:34.498 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 154618822659 -lt 154618822659 2020-09-12T13:57:34.499 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T13:57:34.499 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T13:57:34.499 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T13:57:35.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 1 == 0 2020-09-12T13:57:35.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T13:57:35.036 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T13:57:35.037 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T13:57:35.037 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T13:57:35.037 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T13:57:35.037 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T13:57:35.037 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T13:57:35.439 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T13:57:35.440 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T13:57:35.440 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T13:57:35.440 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T13:57:35.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T13:57:35.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' -1 2020-09-12T13:57:35.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1598: wait_for_clean: loop=0 2020-09-12T13:57:35.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1599: wait_for_clean: num_active_clean=0 2020-09-12T13:57:35.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T13:57:35.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T13:57:36.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T13:57:36.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T13:57:36.073 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T13:57:36.073 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T13:57:36.073 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T13:57:36.073 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T13:57:36.074 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T13:57:36.074 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T13:57:36.468 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T13:57:36.468 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T13:57:36.469 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T13:57:36.469 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T13:57:37.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T13:57:37.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T13:57:37.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T13:57:37.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T13:57:37.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T13:57:37.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T13:57:37.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T13:57:37.001 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T13:57:37.001 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T13:57:37.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T13:57:37.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T13:57:37.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 1 >= 8 )) 2020-09-12T13:57:37.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T13:57:37.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.2 2020-09-12T13:57:37.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T13:57:37.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T13:57:37.733 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T13:57:37.733 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T13:57:37.733 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T13:57:37.734 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T13:57:37.734 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T13:57:37.734 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T13:57:38.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T13:57:38.128 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T13:57:38.128 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T13:57:38.128 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T13:57:38.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T13:57:38.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T13:57:38.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T13:57:38.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T13:57:38.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T13:57:38.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T13:57:38.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T13:57:38.685 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T13:57:38.685 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T13:57:39.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T13:57:39.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T13:57:39.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 2 >= 8 )) 2020-09-12T13:57:39.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T13:57:39.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.4 2020-09-12T13:57:39.616 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T13:57:39.617 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T13:57:39.617 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T13:57:39.617 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T13:57:39.617 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T13:57:39.617 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T13:57:39.618 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T13:57:39.618 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T13:57:40.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T13:57:40.019 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T13:57:40.019 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T13:57:40.020 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T13:57:40.557 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T13:57:40.557 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T13:57:40.557 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T13:57:40.558 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T13:57:40.558 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T13:57:40.558 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T13:57:40.558 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T13:57:40.558 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T13:57:40.558 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T13:57:41.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T13:57:41.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T13:57:41.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 3 >= 8 )) 2020-09-12T13:57:41.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T13:57:41.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.8 2020-09-12T13:57:41.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T13:57:41.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T13:57:41.970 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T13:57:41.971 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T13:57:41.971 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T13:57:41.971 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T13:57:41.971 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T13:57:41.971 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T13:57:42.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T13:57:42.376 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T13:57:42.376 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T13:57:42.377 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T13:57:42.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T13:57:42.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T13:57:42.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T13:57:42.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T13:57:42.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T13:57:42.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T13:57:42.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T13:57:42.919 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T13:57:42.919 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T13:57:43.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T13:57:43.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T13:57:43.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 4 >= 8 )) 2020-09-12T13:57:43.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T13:57:43.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 1.6 2020-09-12T13:57:45.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T13:57:45.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T13:57:45.078 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T13:57:45.078 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T13:57:45.078 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T13:57:45.078 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T13:57:45.079 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T13:57:45.079 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T13:57:45.482 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T13:57:45.483 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T13:57:45.483 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T13:57:45.483 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T13:57:46.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T13:57:46.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T13:57:46.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T13:57:46.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T13:57:46.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T13:57:46.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T13:57:46.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T13:57:46.042 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T13:57:46.042 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T13:57:46.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T13:57:46.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T13:57:46.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 5 >= 8 )) 2020-09-12T13:57:46.583 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T13:57:46.583 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 3.2 2020-09-12T13:57:49.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T13:57:49.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T13:57:49.784 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T13:57:49.784 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T13:57:49.784 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T13:57:49.784 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T13:57:49.785 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T13:57:49.785 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T13:57:50.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T13:57:50.224 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T13:57:50.224 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T13:57:50.225 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T13:57:50.761 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T13:57:50.762 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T13:57:50.762 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T13:57:50.762 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T13:57:50.762 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T13:57:50.762 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T13:57:50.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T13:57:50.763 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T13:57:50.763 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T13:57:51.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T13:57:51.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T13:57:51.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 6 >= 8 )) 2020-09-12T13:57:51.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T13:57:51.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 6.4 2020-09-12T13:57:57.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T13:57:57.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T13:57:57.719 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T13:57:57.719 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T13:57:57.719 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T13:57:57.719 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T13:57:57.720 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T13:57:57.720 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T13:57:58.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T13:57:58.128 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T13:57:58.129 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T13:57:58.129 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T13:57:58.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T13:57:58.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T13:57:58.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T13:57:58.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T13:57:58.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T13:57:58.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T13:57:58.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T13:57:58.676 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T13:57:58.676 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T13:57:59.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T13:57:59.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T13:57:59.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 7 >= 8 )) 2020-09-12T13:57:59.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T13:57:59.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 7.3 2020-09-12T13:58:06.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T13:58:06.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T13:58:06.517 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T13:58:06.518 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T13:58:06.518 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T13:58:06.518 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T13:58:06.518 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T13:58:06.518 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T13:58:06.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T13:58:06.926 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T13:58:06.926 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T13:58:06.926 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T13:58:07.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T13:58:07.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T13:58:07.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T13:58:07.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T13:58:07.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T13:58:07.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T13:58:07.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T13:58:07.464 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T13:58:07.464 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T13:58:08.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T13:58:08.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T13:58:08.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 8 >= 8 )) 2020-09-12T13:58:08.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: wait_for_clean: ceph report 2020-09-12T13:58:08.528 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T13:58:08.528 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_fingerprint": "e6488caa-de1d-4040-9a1d-83b52adca65d", 2020-09-12T13:58:08.528 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "16.0.0-5485-gf346cc73d15", 2020-09-12T13:58:08.529 INFO:tasks.workunit.client.0.smithi027.stdout: "commit": "f346cc73d15a568d3bec1c99a58953de76f57b85", 2020-09-12T13:58:08.529 INFO:tasks.workunit.client.0.smithi027.stdout: "timestamp": "2020-09-12T13:58:08.525989+0000", 2020-09-12T13:58:08.529 INFO:tasks.workunit.client.0.smithi027.stdout: "tag": "", 2020-09-12T13:58:08.530 INFO:tasks.workunit.client.0.smithi027.stdout: "health": { 2020-09-12T13:58:08.530 INFO:tasks.workunit.client.0.smithi027.stdout: "status": "HEALTH_WARN", 2020-09-12T13:58:08.530 INFO:tasks.workunit.client.0.smithi027.stdout: "checks": { 2020-09-12T13:58:08.531 INFO:tasks.workunit.client.0.smithi027.stdout: "OSDMAP_FLAGS": { 2020-09-12T13:58:08.531 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T13:58:08.531 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T13:58:08.531 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "nodown,noout,noin flag(s) set", 2020-09-12T13:58:08.532 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 17 2020-09-12T13:58:08.532 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.532 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [], 2020-09-12T13:58:08.532 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T13:58:08.533 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.533 INFO:tasks.workunit.client.0.smithi027.stdout: "OSD_DOWN": { 2020-09-12T13:58:08.533 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T13:58:08.534 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T13:58:08.534 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "1 osds down", 2020-09-12T13:58:08.534 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 1 2020-09-12T13:58:08.534 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.535 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [ 2020-09-12T13:58:08.535 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.535 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "osd.1 (root=default,host=smithi027) is down" 2020-09-12T13:58:08.536 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.536 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T13:58:08.536 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T13:58:08.536 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.537 INFO:tasks.workunit.client.0.smithi027.stdout: "PG_DEGRADED": { 2020-09-12T13:58:08.537 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T13:58:08.537 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T13:58:08.538 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 1 pg degraded", 2020-09-12T13:58:08.538 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 1 2020-09-12T13:58:08.538 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.539 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [ 2020-09-12T13:58:08.539 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.539 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2020-09-12T13:58:08.539 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.540 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T13:58:08.540 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T13:58:08.540 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.541 INFO:tasks.workunit.client.0.smithi027.stdout: "POOL_APP_NOT_ENABLED": { 2020-09-12T13:58:08.541 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T13:58:08.541 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T13:58:08.541 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "1 pool(s) do not have an application enabled", 2020-09-12T13:58:08.542 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 1 2020-09-12T13:58:08.542 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.542 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [ 2020-09-12T13:58:08.543 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.543 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "application not enabled on pool 'test'" 2020-09-12T13:58:08.543 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.543 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.544 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2020-09-12T13:58:08.544 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.544 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T13:58:08.545 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T13:58:08.545 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.545 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.546 INFO:tasks.workunit.client.0.smithi027.stdout: "mutes": [] 2020-09-12T13:58:08.546 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.546 INFO:tasks.workunit.client.0.smithi027.stdout: "monmap_first_committed": 1, 2020-09-12T13:58:08.546 INFO:tasks.workunit.client.0.smithi027.stdout: "monmap_last_committed": 1, 2020-09-12T13:58:08.546 INFO:tasks.workunit.client.0.smithi027.stdout: "monmap": { 2020-09-12T13:58:08.546 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 1, 2020-09-12T13:58:08.547 INFO:tasks.workunit.client.0.smithi027.stdout: "fsid": "7e90bad3-7973-4a8c-be65-6326ab931fe8", 2020-09-12T13:58:08.547 INFO:tasks.workunit.client.0.smithi027.stdout: "modified": "2020-09-12T13:56:12.146949Z", 2020-09-12T13:58:08.547 INFO:tasks.workunit.client.0.smithi027.stdout: "created": "2020-09-12T13:56:12.146949Z", 2020-09-12T13:58:08.547 INFO:tasks.workunit.client.0.smithi027.stdout: "min_mon_release": 16, 2020-09-12T13:58:08.547 INFO:tasks.workunit.client.0.smithi027.stdout: "min_mon_release_name": "pacific", 2020-09-12T13:58:08.547 INFO:tasks.workunit.client.0.smithi027.stdout: "features": { 2020-09-12T13:58:08.547 INFO:tasks.workunit.client.0.smithi027.stdout: "persistent": [ 2020-09-12T13:58:08.548 INFO:tasks.workunit.client.0.smithi027.stdout: "kraken", 2020-09-12T13:58:08.548 INFO:tasks.workunit.client.0.smithi027.stdout: "luminous", 2020-09-12T13:58:08.548 INFO:tasks.workunit.client.0.smithi027.stdout: "mimic", 2020-09-12T13:58:08.548 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap-prune", 2020-09-12T13:58:08.548 INFO:tasks.workunit.client.0.smithi027.stdout: "nautilus", 2020-09-12T13:58:08.548 INFO:tasks.workunit.client.0.smithi027.stdout: "octopus", 2020-09-12T13:58:08.548 INFO:tasks.workunit.client.0.smithi027.stdout: "pacific" 2020-09-12T13:58:08.548 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T13:58:08.549 INFO:tasks.workunit.client.0.smithi027.stdout: "optional": [] 2020-09-12T13:58:08.549 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.549 INFO:tasks.workunit.client.0.smithi027.stdout: "mons": [ 2020-09-12T13:58:08.549 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.549 INFO:tasks.workunit.client.0.smithi027.stdout: "rank": 0, 2020-09-12T13:58:08.549 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "a", 2020-09-12T13:58:08.549 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T13:58:08.550 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T13:58:08.550 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.550 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T13:58:08.550 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:7115", 2020-09-12T13:58:08.550 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 0 2020-09-12T13:58:08.550 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.550 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.550 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.551 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:7115/0", 2020-09-12T13:58:08.551 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:7115/0", 2020-09-12T13:58:08.551 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 0, 2020-09-12T13:58:08.551 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 0 2020-09-12T13:58:08.551 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.551 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.551 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.551 INFO:tasks.workunit.client.0.smithi027.stdout: "quorum": [ 2020-09-12T13:58:08.552 INFO:tasks.workunit.client.0.smithi027.stdout: 0 2020-09-12T13:58:08.552 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T13:58:08.552 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap": { 2020-09-12T13:58:08.552 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 41, 2020-09-12T13:58:08.552 INFO:tasks.workunit.client.0.smithi027.stdout: "fsid": "7e90bad3-7973-4a8c-be65-6326ab931fe8", 2020-09-12T13:58:08.552 INFO:tasks.workunit.client.0.smithi027.stdout: "created": "2020-09-12T13:56:12.939531+0000", 2020-09-12T13:58:08.552 INFO:tasks.workunit.client.0.smithi027.stdout: "modified": "2020-09-12T13:57:31.409814+0000", 2020-09-12T13:58:08.553 INFO:tasks.workunit.client.0.smithi027.stdout: "last_up_change": "2020-09-12T13:57:29.603781+0000", 2020-09-12T13:58:08.553 INFO:tasks.workunit.client.0.smithi027.stdout: "last_in_change": "2020-09-12T13:57:30.999416+0000", 2020-09-12T13:58:08.553 INFO:tasks.workunit.client.0.smithi027.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-09-12T13:58:08.553 INFO:tasks.workunit.client.0.smithi027.stdout: "flags_num": 5800384, 2020-09-12T13:58:08.553 INFO:tasks.workunit.client.0.smithi027.stdout: "flags_set": [ 2020-09-12T13:58:08.553 INFO:tasks.workunit.client.0.smithi027.stdout: "nodown", 2020-09-12T13:58:08.553 INFO:tasks.workunit.client.0.smithi027.stdout: "noin", 2020-09-12T13:58:08.553 INFO:tasks.workunit.client.0.smithi027.stdout: "noout", 2020-09-12T13:58:08.554 INFO:tasks.workunit.client.0.smithi027.stdout: "pglog_hardlimit", 2020-09-12T13:58:08.554 INFO:tasks.workunit.client.0.smithi027.stdout: "purged_snapdirs", 2020-09-12T13:58:08.554 INFO:tasks.workunit.client.0.smithi027.stdout: "recovery_deletes", 2020-09-12T13:58:08.554 INFO:tasks.workunit.client.0.smithi027.stdout: "sortbitwise" 2020-09-12T13:58:08.554 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T13:58:08.554 INFO:tasks.workunit.client.0.smithi027.stdout: "crush_version": 7, 2020-09-12T13:58:08.554 INFO:tasks.workunit.client.0.smithi027.stdout: "full_ratio": 0.99000000953674316, 2020-09-12T13:58:08.554 INFO:tasks.workunit.client.0.smithi027.stdout: "backfillfull_ratio": 0.99000000953674316, 2020-09-12T13:58:08.555 INFO:tasks.workunit.client.0.smithi027.stdout: "nearfull_ratio": 0.85000002384185791, 2020-09-12T13:58:08.555 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_snapshot": "", 2020-09-12T13:58:08.555 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_max": 1, 2020-09-12T13:58:08.555 INFO:tasks.workunit.client.0.smithi027.stdout: "max_osd": 3, 2020-09-12T13:58:08.555 INFO:tasks.workunit.client.0.smithi027.stdout: "require_min_compat_client": "luminous", 2020-09-12T13:58:08.555 INFO:tasks.workunit.client.0.smithi027.stdout: "min_compat_client": "jewel", 2020-09-12T13:58:08.555 INFO:tasks.workunit.client.0.smithi027.stdout: "require_osd_release": "pacific", 2020-09-12T13:58:08.556 INFO:tasks.workunit.client.0.smithi027.stdout: "pools": [ 2020-09-12T13:58:08.556 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.556 INFO:tasks.workunit.client.0.smithi027.stdout: "pool": 1, 2020-09-12T13:58:08.556 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_name": "test", 2020-09-12T13:58:08.556 INFO:tasks.workunit.client.0.smithi027.stdout: "create_time": "2020-09-12T13:56:39.309715+0000", 2020-09-12T13:58:08.556 INFO:tasks.workunit.client.0.smithi027.stdout: "flags": 1, 2020-09-12T13:58:08.556 INFO:tasks.workunit.client.0.smithi027.stdout: "flags_names": "hashpspool", 2020-09-12T13:58:08.556 INFO:tasks.workunit.client.0.smithi027.stdout: "type": 1, 2020-09-12T13:58:08.557 INFO:tasks.workunit.client.0.smithi027.stdout: "size": 3, 2020-09-12T13:58:08.557 INFO:tasks.workunit.client.0.smithi027.stdout: "min_size": 2, 2020-09-12T13:58:08.557 INFO:tasks.workunit.client.0.smithi027.stdout: "crush_rule": 0, 2020-09-12T13:58:08.557 INFO:tasks.workunit.client.0.smithi027.stdout: "object_hash": 2, 2020-09-12T13:58:08.557 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_autoscale_mode": "off", 2020-09-12T13:58:08.557 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_num": 1, 2020-09-12T13:58:08.557 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_placement_num": 1, 2020-09-12T13:58:08.557 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_placement_num_target": 1, 2020-09-12T13:58:08.558 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_num_target": 1, 2020-09-12T13:58:08.558 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_num_pending": 1, 2020-09-12T13:58:08.558 INFO:tasks.workunit.client.0.smithi027.stdout: "last_pg_merge_meta": { 2020-09-12T13:58:08.558 INFO:tasks.workunit.client.0.smithi027.stdout: "source_pgid": "0.0", 2020-09-12T13:58:08.558 INFO:tasks.workunit.client.0.smithi027.stdout: "ready_epoch": 0, 2020-09-12T13:58:08.558 INFO:tasks.workunit.client.0.smithi027.stdout: "last_epoch_started": 0, 2020-09-12T13:58:08.558 INFO:tasks.workunit.client.0.smithi027.stdout: "last_epoch_clean": 0, 2020-09-12T13:58:08.559 INFO:tasks.workunit.client.0.smithi027.stdout: "source_version": "0'0", 2020-09-12T13:58:08.559 INFO:tasks.workunit.client.0.smithi027.stdout: "target_version": "0'0" 2020-09-12T13:58:08.559 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.559 INFO:tasks.workunit.client.0.smithi027.stdout: "last_change": "29", 2020-09-12T13:58:08.559 INFO:tasks.workunit.client.0.smithi027.stdout: "last_force_op_resend": "0", 2020-09-12T13:58:08.559 INFO:tasks.workunit.client.0.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2020-09-12T13:58:08.559 INFO:tasks.workunit.client.0.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2020-09-12T13:58:08.559 INFO:tasks.workunit.client.0.smithi027.stdout: "auid": 0, 2020-09-12T13:58:08.560 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_mode": "selfmanaged", 2020-09-12T13:58:08.560 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_seq": 0, 2020-09-12T13:58:08.560 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_epoch": 0, 2020-09-12T13:58:08.560 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_snaps": [], 2020-09-12T13:58:08.560 INFO:tasks.workunit.client.0.smithi027.stdout: "removed_snaps": "[]", 2020-09-12T13:58:08.560 INFO:tasks.workunit.client.0.smithi027.stdout: "quota_max_bytes": 0, 2020-09-12T13:58:08.560 INFO:tasks.workunit.client.0.smithi027.stdout: "quota_max_objects": 0, 2020-09-12T13:58:08.560 INFO:tasks.workunit.client.0.smithi027.stdout: "tiers": [], 2020-09-12T13:58:08.561 INFO:tasks.workunit.client.0.smithi027.stdout: "tier_of": -1, 2020-09-12T13:58:08.561 INFO:tasks.workunit.client.0.smithi027.stdout: "read_tier": -1, 2020-09-12T13:58:08.561 INFO:tasks.workunit.client.0.smithi027.stdout: "write_tier": -1, 2020-09-12T13:58:08.561 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_mode": "none", 2020-09-12T13:58:08.561 INFO:tasks.workunit.client.0.smithi027.stdout: "target_max_bytes": 0, 2020-09-12T13:58:08.561 INFO:tasks.workunit.client.0.smithi027.stdout: "target_max_objects": 0, 2020-09-12T13:58:08.561 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-09-12T13:58:08.561 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-09-12T13:58:08.562 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2020-09-12T13:58:08.562 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_min_flush_age": 0, 2020-09-12T13:58:08.562 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_min_evict_age": 0, 2020-09-12T13:58:08.562 INFO:tasks.workunit.client.0.smithi027.stdout: "erasure_code_profile": "", 2020-09-12T13:58:08.562 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_params": { 2020-09-12T13:58:08.562 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "none" 2020-09-12T13:58:08.562 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.562 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_period": 0, 2020-09-12T13:58:08.563 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_count": 0, 2020-09-12T13:58:08.563 INFO:tasks.workunit.client.0.smithi027.stdout: "use_gmt_hitset": true, 2020-09-12T13:58:08.563 INFO:tasks.workunit.client.0.smithi027.stdout: "min_read_recency_for_promote": 0, 2020-09-12T13:58:08.563 INFO:tasks.workunit.client.0.smithi027.stdout: "min_write_recency_for_promote": 0, 2020-09-12T13:58:08.563 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2020-09-12T13:58:08.563 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_search_last_n": 0, 2020-09-12T13:58:08.563 INFO:tasks.workunit.client.0.smithi027.stdout: "grade_table": [], 2020-09-12T13:58:08.564 INFO:tasks.workunit.client.0.smithi027.stdout: "stripe_width": 0, 2020-09-12T13:58:08.564 INFO:tasks.workunit.client.0.smithi027.stdout: "expected_num_objects": 0, 2020-09-12T13:58:08.564 INFO:tasks.workunit.client.0.smithi027.stdout: "fast_read": false, 2020-09-12T13:58:08.564 INFO:tasks.workunit.client.0.smithi027.stdout: "options": {}, 2020-09-12T13:58:08.564 INFO:tasks.workunit.client.0.smithi027.stdout: "application_metadata": {} 2020-09-12T13:58:08.564 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.564 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T13:58:08.564 INFO:tasks.workunit.client.0.smithi027.stdout: "osds": [ 2020-09-12T13:58:08.565 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.565 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T13:58:08.565 INFO:tasks.workunit.client.0.smithi027.stdout: "uuid": "97c8d704-849c-44e1-a64d-c66c37f051bf", 2020-09-12T13:58:08.565 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 1, 2020-09-12T13:58:08.565 INFO:tasks.workunit.client.0.smithi027.stdout: "in": 1, 2020-09-12T13:58:08.565 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 1, 2020-09-12T13:58:08.565 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_affinity": 1, 2020-09-12T13:58:08.565 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_begin": 5, 2020-09-12T13:58:08.566 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_end": 29, 2020-09-12T13:58:08.566 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 35, 2020-09-12T13:58:08.566 INFO:tasks.workunit.client.0.smithi027.stdout: "up_thru": 39, 2020-09-12T13:58:08.566 INFO:tasks.workunit.client.0.smithi027.stdout: "down_at": 30, 2020-09-12T13:58:08.566 INFO:tasks.workunit.client.0.smithi027.stdout: "lost_at": 0, 2020-09-12T13:58:08.566 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T13:58:08.566 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T13:58:08.567 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.567 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T13:58:08.567 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6802", 2020-09-12T13:58:08.567 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 27534 2020-09-12T13:58:08.567 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.567 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.567 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T13:58:08.567 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6803", 2020-09-12T13:58:08.568 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 27534 2020-09-12T13:58:08.568 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.568 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.568 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.568 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addrs": { 2020-09-12T13:58:08.568 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T13:58:08.568 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.569 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T13:58:08.569 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6804", 2020-09-12T13:58:08.569 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 27534 2020-09-12T13:58:08.569 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.569 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.569 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T13:58:08.569 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6805", 2020-09-12T13:58:08.569 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 27534 2020-09-12T13:58:08.570 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.570 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.570 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.570 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addrs": { 2020-09-12T13:58:08.570 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T13:58:08.570 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.570 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T13:58:08.570 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6808", 2020-09-12T13:58:08.571 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 27534 2020-09-12T13:58:08.571 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.571 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.571 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T13:58:08.571 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6809", 2020-09-12T13:58:08.571 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 27534 2020-09-12T13:58:08.571 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.572 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.572 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.572 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addrs": { 2020-09-12T13:58:08.572 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T13:58:08.572 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.572 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T13:58:08.572 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6806", 2020-09-12T13:58:08.572 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 27534 2020-09-12T13:58:08.573 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.573 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.573 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T13:58:08.573 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6807", 2020-09-12T13:58:08.573 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 27534 2020-09-12T13:58:08.573 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.573 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.573 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.574 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:6803/27534", 2020-09-12T13:58:08.574 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addr": "127.0.0.1:6805/27534", 2020-09-12T13:58:08.574 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addr": "127.0.0.1:6809/27534", 2020-09-12T13:58:08.574 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addr": "127.0.0.1:6807/27534", 2020-09-12T13:58:08.574 INFO:tasks.workunit.client.0.smithi027.stdout: "state": [ 2020-09-12T13:58:08.574 INFO:tasks.workunit.client.0.smithi027.stdout: "exists", 2020-09-12T13:58:08.574 INFO:tasks.workunit.client.0.smithi027.stdout: "up" 2020-09-12T13:58:08.574 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.575 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.575 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.575 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T13:58:08.575 INFO:tasks.workunit.client.0.smithi027.stdout: "uuid": "96117818-e665-481f-9e92-b70e5d147a31", 2020-09-12T13:58:08.575 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 0, 2020-09-12T13:58:08.575 INFO:tasks.workunit.client.0.smithi027.stdout: "in": 1, 2020-09-12T13:58:08.575 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 1, 2020-09-12T13:58:08.575 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_affinity": 1, 2020-09-12T13:58:08.576 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_begin": 0, 2020-09-12T13:58:08.576 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_end": 0, 2020-09-12T13:58:08.576 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 10, 2020-09-12T13:58:08.576 INFO:tasks.workunit.client.0.smithi027.stdout: "up_thru": 23, 2020-09-12T13:58:08.576 INFO:tasks.workunit.client.0.smithi027.stdout: "down_at": 31, 2020-09-12T13:58:08.576 INFO:tasks.workunit.client.0.smithi027.stdout: "lost_at": 0, 2020-09-12T13:58:08.576 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T13:58:08.576 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T13:58:08.577 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.577 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T13:58:08.577 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6810", 2020-09-12T13:58:08.577 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 22674 2020-09-12T13:58:08.577 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.577 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.577 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T13:58:08.578 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6811", 2020-09-12T13:58:08.578 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 22674 2020-09-12T13:58:08.578 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.578 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.578 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.578 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addrs": { 2020-09-12T13:58:08.578 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T13:58:08.578 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.579 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T13:58:08.579 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6812", 2020-09-12T13:58:08.579 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 22674 2020-09-12T13:58:08.579 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.579 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.579 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T13:58:08.579 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6813", 2020-09-12T13:58:08.579 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 22674 2020-09-12T13:58:08.580 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.580 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.580 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.580 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addrs": { 2020-09-12T13:58:08.580 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T13:58:08.580 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.580 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T13:58:08.580 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6816", 2020-09-12T13:58:08.581 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 22674 2020-09-12T13:58:08.581 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.581 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.581 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T13:58:08.581 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6817", 2020-09-12T13:58:08.581 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 22674 2020-09-12T13:58:08.581 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.581 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.582 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.582 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addrs": { 2020-09-12T13:58:08.582 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T13:58:08.582 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.582 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T13:58:08.582 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6814", 2020-09-12T13:58:08.582 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 22674 2020-09-12T13:58:08.582 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.583 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.583 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T13:58:08.583 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6815", 2020-09-12T13:58:08.583 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 22674 2020-09-12T13:58:08.583 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.583 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.583 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.584 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:6811/22674", 2020-09-12T13:58:08.584 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addr": "127.0.0.1:6813/22674", 2020-09-12T13:58:08.584 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addr": "127.0.0.1:6817/22674", 2020-09-12T13:58:08.584 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addr": "127.0.0.1:6815/22674", 2020-09-12T13:58:08.584 INFO:tasks.workunit.client.0.smithi027.stdout: "state": [ 2020-09-12T13:58:08.584 INFO:tasks.workunit.client.0.smithi027.stdout: "exists" 2020-09-12T13:58:08.584 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.584 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.585 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.585 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T13:58:08.585 INFO:tasks.workunit.client.0.smithi027.stdout: "uuid": "cadcbbd5-8cd1-4081-9e23-e263c65059d2", 2020-09-12T13:58:08.585 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 1, 2020-09-12T13:58:08.585 INFO:tasks.workunit.client.0.smithi027.stdout: "in": 1, 2020-09-12T13:58:08.585 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 1, 2020-09-12T13:58:08.585 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_affinity": 1, 2020-09-12T13:58:08.585 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_begin": 15, 2020-09-12T13:58:08.586 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_end": 29, 2020-09-12T13:58:08.586 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 36, 2020-09-12T13:58:08.586 INFO:tasks.workunit.client.0.smithi027.stdout: "up_thru": 0, 2020-09-12T13:58:08.586 INFO:tasks.workunit.client.0.smithi027.stdout: "down_at": 32, 2020-09-12T13:58:08.586 INFO:tasks.workunit.client.0.smithi027.stdout: "lost_at": 0, 2020-09-12T13:58:08.586 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T13:58:08.586 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T13:58:08.586 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.587 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T13:58:08.587 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6810", 2020-09-12T13:58:08.587 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 27933 2020-09-12T13:58:08.587 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.587 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.587 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T13:58:08.587 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6811", 2020-09-12T13:58:08.587 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 27933 2020-09-12T13:58:08.588 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.588 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.588 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.588 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addrs": { 2020-09-12T13:58:08.588 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T13:58:08.588 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.588 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T13:58:08.588 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6812", 2020-09-12T13:58:08.589 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 27933 2020-09-12T13:58:08.589 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.589 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.589 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T13:58:08.589 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6813", 2020-09-12T13:58:08.589 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 27933 2020-09-12T13:58:08.589 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.589 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.590 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.590 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addrs": { 2020-09-12T13:58:08.590 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T13:58:08.590 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.590 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T13:58:08.590 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6816", 2020-09-12T13:58:08.590 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 27933 2020-09-12T13:58:08.591 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.591 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.591 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T13:58:08.591 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6817", 2020-09-12T13:58:08.591 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 27933 2020-09-12T13:58:08.591 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.591 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.591 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.592 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addrs": { 2020-09-12T13:58:08.592 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T13:58:08.592 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.592 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T13:58:08.592 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6814", 2020-09-12T13:58:08.592 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 27933 2020-09-12T13:58:08.592 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.592 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.593 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T13:58:08.593 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6815", 2020-09-12T13:58:08.593 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 27933 2020-09-12T13:58:08.593 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.593 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.593 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.593 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:6811/27933", 2020-09-12T13:58:08.593 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addr": "127.0.0.1:6813/27933", 2020-09-12T13:58:08.594 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addr": "127.0.0.1:6817/27933", 2020-09-12T13:58:08.594 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addr": "127.0.0.1:6815/27933", 2020-09-12T13:58:08.594 INFO:tasks.workunit.client.0.smithi027.stdout: "state": [ 2020-09-12T13:58:08.594 INFO:tasks.workunit.client.0.smithi027.stdout: "exists", 2020-09-12T13:58:08.594 INFO:tasks.workunit.client.0.smithi027.stdout: "up" 2020-09-12T13:58:08.594 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.594 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.594 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T13:58:08.595 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_xinfo": [ 2020-09-12T13:58:08.595 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.595 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T13:58:08.595 INFO:tasks.workunit.client.0.smithi027.stdout: "down_stamp": "2020-09-12T13:57:22.526794+0000", 2020-09-12T13:58:08.595 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_probability": 0, 2020-09-12T13:58:08.595 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_interval": 0, 2020-09-12T13:58:08.595 INFO:tasks.workunit.client.0.smithi027.stdout: "features": 4540138292841938943, 2020-09-12T13:58:08.596 INFO:tasks.workunit.client.0.smithi027.stdout: "old_weight": 0, 2020-09-12T13:58:08.596 INFO:tasks.workunit.client.0.smithi027.stdout: "last_purged_snaps_scrub": "2020-09-12T13:56:21.443431+0000", 2020-09-12T13:58:08.596 INFO:tasks.workunit.client.0.smithi027.stdout: "dead_epoch": 0 2020-09-12T13:58:08.596 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.596 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.596 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T13:58:08.596 INFO:tasks.workunit.client.0.smithi027.stdout: "down_stamp": "2020-09-12T13:57:23.063642+0000", 2020-09-12T13:58:08.596 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_probability": 0, 2020-09-12T13:58:08.597 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_interval": 0, 2020-09-12T13:58:08.597 INFO:tasks.workunit.client.0.smithi027.stdout: "features": 4540138292841938943, 2020-09-12T13:58:08.597 INFO:tasks.workunit.client.0.smithi027.stdout: "old_weight": 0, 2020-09-12T13:58:08.597 INFO:tasks.workunit.client.0.smithi027.stdout: "last_purged_snaps_scrub": "2020-09-12T13:56:29.126668+0000", 2020-09-12T13:58:08.597 INFO:tasks.workunit.client.0.smithi027.stdout: "dead_epoch": 0 2020-09-12T13:58:08.597 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.597 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.597 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T13:58:08.598 INFO:tasks.workunit.client.0.smithi027.stdout: "down_stamp": "2020-09-12T13:57:23.633570+0000", 2020-09-12T13:58:08.598 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_probability": 0, 2020-09-12T13:58:08.598 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_interval": 0, 2020-09-12T13:58:08.598 INFO:tasks.workunit.client.0.smithi027.stdout: "features": 4540138292841938943, 2020-09-12T13:58:08.598 INFO:tasks.workunit.client.0.smithi027.stdout: "old_weight": 0, 2020-09-12T13:58:08.598 INFO:tasks.workunit.client.0.smithi027.stdout: "last_purged_snaps_scrub": "2020-09-12T13:56:36.729546+0000", 2020-09-12T13:58:08.598 INFO:tasks.workunit.client.0.smithi027.stdout: "dead_epoch": 0 2020-09-12T13:58:08.598 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.599 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T13:58:08.599 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_upmap": [], 2020-09-12T13:58:08.599 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_upmap_items": [], 2020-09-12T13:58:08.599 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_temp": [], 2020-09-12T13:58:08.599 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_temp": [], 2020-09-12T13:58:08.599 INFO:tasks.workunit.client.0.smithi027.stdout: "blocklist": {}, 2020-09-12T13:58:08.599 INFO:tasks.workunit.client.0.smithi027.stdout: "erasure_code_profiles": { 2020-09-12T13:58:08.600 INFO:tasks.workunit.client.0.smithi027.stdout: "default": { 2020-09-12T13:58:08.600 INFO:tasks.workunit.client.0.smithi027.stdout: "k": "2", 2020-09-12T13:58:08.600 INFO:tasks.workunit.client.0.smithi027.stdout: "m": "2", 2020-09-12T13:58:08.600 INFO:tasks.workunit.client.0.smithi027.stdout: "plugin": "jerasure", 2020-09-12T13:58:08.600 INFO:tasks.workunit.client.0.smithi027.stdout: "technique": "reed_sol_van" 2020-09-12T13:58:08.600 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.600 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.600 INFO:tasks.workunit.client.0.smithi027.stdout: "removed_snaps_queue": [], 2020-09-12T13:58:08.601 INFO:tasks.workunit.client.0.smithi027.stdout: "new_removed_snaps": [], 2020-09-12T13:58:08.601 INFO:tasks.workunit.client.0.smithi027.stdout: "new_purged_snaps": [], 2020-09-12T13:58:08.601 INFO:tasks.workunit.client.0.smithi027.stdout: "crush_node_flags": {}, 2020-09-12T13:58:08.601 INFO:tasks.workunit.client.0.smithi027.stdout: "device_class_flags": {} 2020-09-12T13:58:08.601 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.601 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_metadata": [ 2020-09-12T13:58:08.601 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.601 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T13:58:08.602 INFO:tasks.workunit.client.0.smithi027.stdout: "arch": "x86_64", 2020-09-12T13:58:08.602 INFO:tasks.workunit.client.0.smithi027.stdout: "back_addr": "[v2:127.0.0.1:6804/27534,v1:127.0.0.1:6805/27534]", 2020-09-12T13:58:08.602 INFO:tasks.workunit.client.0.smithi027.stdout: "back_iface": "", 2020-09-12T13:58:08.602 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs": "1", 2020-09-12T13:58:08.602 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_db": "0", 2020-09-12T13:58:08.602 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_wal": "0", 2020-09-12T13:58:08.602 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_single_shared_device": "1", 2020-09-12T13:58:08.602 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_access_mode": "file", 2020-09-12T13:58:08.603 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_block_size": "4096", 2020-09-12T13:58:08.603 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_devices": "sda", 2020-09-12T13:58:08.603 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_driver": "KernelDevice", 2020-09-12T13:58:08.603 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2020-09-12T13:58:08.603 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_rotational": "1", 2020-09-12T13:58:08.603 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_size": "107374182400", 2020-09-12T13:58:08.603 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_support_discard": "0", 2020-09-12T13:58:08.603 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_type": "hdd", 2020-09-12T13:58:08.604 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_release": "pacific", 2020-09-12T13:58:08.604 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version": "ceph version 16.0.0-5485-gf346cc73d15 (f346cc73d15a568d3bec1c99a58953de76f57b85) pacific (dev)", 2020-09-12T13:58:08.604 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version_short": "16.0.0-5485-gf346cc73d15", 2020-09-12T13:58:08.604 INFO:tasks.workunit.client.0.smithi027.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2020-09-12T13:58:08.604 INFO:tasks.workunit.client.0.smithi027.stdout: "default_device_class": "hdd", 2020-09-12T13:58:08.604 INFO:tasks.workunit.client.0.smithi027.stdout: "device_ids": "sda=ST1000NM0033-9ZM173_Z1W4H3N7", 2020-09-12T13:58:08.604 INFO:tasks.workunit.client.0.smithi027.stdout: "device_paths": "sda=/dev/disk/by-path/pci-0000:00:1f.2-ata-1", 2020-09-12T13:58:08.604 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": "sda", 2020-09-12T13:58:08.605 INFO:tasks.workunit.client.0.smithi027.stdout: "distro": "centos", 2020-09-12T13:58:08.605 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_description": "CentOS Linux 8 (Core)", 2020-09-12T13:58:08.605 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_version": "8", 2020-09-12T13:58:08.605 INFO:tasks.workunit.client.0.smithi027.stdout: "front_addr": "[v2:127.0.0.1:6802/27534,v1:127.0.0.1:6803/27534]", 2020-09-12T13:58:08.605 INFO:tasks.workunit.client.0.smithi027.stdout: "front_iface": "", 2020-09-12T13:58:08.605 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/27534,v1:127.0.0.1:6809/27534]", 2020-09-12T13:58:08.605 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/27534,v1:127.0.0.1:6807/27534]", 2020-09-12T13:58:08.606 INFO:tasks.workunit.client.0.smithi027.stdout: "hostname": "smithi027", 2020-09-12T13:58:08.606 INFO:tasks.workunit.client.0.smithi027.stdout: "journal_rotational": "1", 2020-09-12T13:58:08.606 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_description": "#1 SMP Sun Jul 26 03:54:29 UTC 2020", 2020-09-12T13:58:08.606 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_version": "4.18.0-193.14.2.el8_2.x86_64", 2020-09-12T13:58:08.606 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_swap_kb": "0", 2020-09-12T13:58:08.606 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_total_kb": "32659336", 2020-09-12T13:58:08.606 INFO:tasks.workunit.client.0.smithi027.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2020-09-12T13:58:08.606 INFO:tasks.workunit.client.0.smithi027.stdout: "objectstore_numa_unknown_devices": "sda", 2020-09-12T13:58:08.607 INFO:tasks.workunit.client.0.smithi027.stdout: "os": "Linux", 2020-09-12T13:58:08.607 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_data": "td/divergent-priors/0", 2020-09-12T13:58:08.607 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_objectstore": "bluestore", 2020-09-12T13:58:08.607 INFO:tasks.workunit.client.0.smithi027.stdout: "osdspec_affinity": "", 2020-09-12T13:58:08.607 INFO:tasks.workunit.client.0.smithi027.stdout: "rotational": "1" 2020-09-12T13:58:08.607 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.607 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.607 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T13:58:08.608 INFO:tasks.workunit.client.0.smithi027.stdout: "arch": "x86_64", 2020-09-12T13:58:08.608 INFO:tasks.workunit.client.0.smithi027.stdout: "back_addr": "[v2:127.0.0.1:6812/22674,v1:127.0.0.1:6813/22674]", 2020-09-12T13:58:08.608 INFO:tasks.workunit.client.0.smithi027.stdout: "back_iface": "", 2020-09-12T13:58:08.608 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs": "1", 2020-09-12T13:58:08.608 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_db": "0", 2020-09-12T13:58:08.608 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_wal": "0", 2020-09-12T13:58:08.608 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_single_shared_device": "1", 2020-09-12T13:58:08.608 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_access_mode": "file", 2020-09-12T13:58:08.609 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_block_size": "4096", 2020-09-12T13:58:08.609 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_devices": "sda", 2020-09-12T13:58:08.609 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_driver": "KernelDevice", 2020-09-12T13:58:08.609 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2020-09-12T13:58:08.609 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_rotational": "1", 2020-09-12T13:58:08.609 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_size": "107374182400", 2020-09-12T13:58:08.609 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_support_discard": "0", 2020-09-12T13:58:08.610 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_type": "hdd", 2020-09-12T13:58:08.610 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_release": "pacific", 2020-09-12T13:58:08.610 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version": "ceph version 16.0.0-5485-gf346cc73d15 (f346cc73d15a568d3bec1c99a58953de76f57b85) pacific (dev)", 2020-09-12T13:58:08.610 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version_short": "16.0.0-5485-gf346cc73d15", 2020-09-12T13:58:08.610 INFO:tasks.workunit.client.0.smithi027.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2020-09-12T13:58:08.610 INFO:tasks.workunit.client.0.smithi027.stdout: "default_device_class": "hdd", 2020-09-12T13:58:08.610 INFO:tasks.workunit.client.0.smithi027.stdout: "device_ids": "sda=ST1000NM0033-9ZM173_Z1W4H3N7", 2020-09-12T13:58:08.610 INFO:tasks.workunit.client.0.smithi027.stdout: "device_paths": "sda=/dev/disk/by-path/pci-0000:00:1f.2-ata-1", 2020-09-12T13:58:08.611 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": "sda", 2020-09-12T13:58:08.611 INFO:tasks.workunit.client.0.smithi027.stdout: "distro": "centos", 2020-09-12T13:58:08.611 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_description": "CentOS Linux 8 (Core)", 2020-09-12T13:58:08.611 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_version": "8", 2020-09-12T13:58:08.611 INFO:tasks.workunit.client.0.smithi027.stdout: "front_addr": "[v2:127.0.0.1:6810/22674,v1:127.0.0.1:6811/22674]", 2020-09-12T13:58:08.611 INFO:tasks.workunit.client.0.smithi027.stdout: "front_iface": "", 2020-09-12T13:58:08.611 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/22674,v1:127.0.0.1:6817/22674]", 2020-09-12T13:58:08.611 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/22674,v1:127.0.0.1:6815/22674]", 2020-09-12T13:58:08.612 INFO:tasks.workunit.client.0.smithi027.stdout: "hostname": "smithi027", 2020-09-12T13:58:08.612 INFO:tasks.workunit.client.0.smithi027.stdout: "journal_rotational": "1", 2020-09-12T13:58:08.612 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_description": "#1 SMP Sun Jul 26 03:54:29 UTC 2020", 2020-09-12T13:58:08.612 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_version": "4.18.0-193.14.2.el8_2.x86_64", 2020-09-12T13:58:08.612 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_swap_kb": "0", 2020-09-12T13:58:08.612 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_total_kb": "32659336", 2020-09-12T13:58:08.612 INFO:tasks.workunit.client.0.smithi027.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2020-09-12T13:58:08.613 INFO:tasks.workunit.client.0.smithi027.stdout: "objectstore_numa_unknown_devices": "sda", 2020-09-12T13:58:08.613 INFO:tasks.workunit.client.0.smithi027.stdout: "os": "Linux", 2020-09-12T13:58:08.613 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_data": "td/divergent-priors/1", 2020-09-12T13:58:08.613 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_objectstore": "bluestore", 2020-09-12T13:58:08.613 INFO:tasks.workunit.client.0.smithi027.stdout: "osdspec_affinity": "", 2020-09-12T13:58:08.613 INFO:tasks.workunit.client.0.smithi027.stdout: "rotational": "1" 2020-09-12T13:58:08.613 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.613 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.614 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T13:58:08.614 INFO:tasks.workunit.client.0.smithi027.stdout: "arch": "x86_64", 2020-09-12T13:58:08.614 INFO:tasks.workunit.client.0.smithi027.stdout: "back_addr": "[v2:127.0.0.1:6812/27933,v1:127.0.0.1:6813/27933]", 2020-09-12T13:58:08.614 INFO:tasks.workunit.client.0.smithi027.stdout: "back_iface": "", 2020-09-12T13:58:08.614 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs": "1", 2020-09-12T13:58:08.614 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_db": "0", 2020-09-12T13:58:08.615 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_wal": "0", 2020-09-12T13:58:08.615 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_single_shared_device": "1", 2020-09-12T13:58:08.615 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_access_mode": "file", 2020-09-12T13:58:08.615 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_block_size": "4096", 2020-09-12T13:58:08.615 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_devices": "sda", 2020-09-12T13:58:08.615 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_driver": "KernelDevice", 2020-09-12T13:58:08.615 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2020-09-12T13:58:08.615 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_rotational": "1", 2020-09-12T13:58:08.616 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_size": "107374182400", 2020-09-12T13:58:08.616 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_support_discard": "0", 2020-09-12T13:58:08.616 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_type": "hdd", 2020-09-12T13:58:08.616 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_release": "pacific", 2020-09-12T13:58:08.616 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version": "ceph version 16.0.0-5485-gf346cc73d15 (f346cc73d15a568d3bec1c99a58953de76f57b85) pacific (dev)", 2020-09-12T13:58:08.616 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version_short": "16.0.0-5485-gf346cc73d15", 2020-09-12T13:58:08.616 INFO:tasks.workunit.client.0.smithi027.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2020-09-12T13:58:08.616 INFO:tasks.workunit.client.0.smithi027.stdout: "default_device_class": "hdd", 2020-09-12T13:58:08.617 INFO:tasks.workunit.client.0.smithi027.stdout: "device_ids": "sda=ST1000NM0033-9ZM173_Z1W4H3N7", 2020-09-12T13:58:08.617 INFO:tasks.workunit.client.0.smithi027.stdout: "device_paths": "sda=/dev/disk/by-path/pci-0000:00:1f.2-ata-1", 2020-09-12T13:58:08.617 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": "sda", 2020-09-12T13:58:08.617 INFO:tasks.workunit.client.0.smithi027.stdout: "distro": "centos", 2020-09-12T13:58:08.617 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_description": "CentOS Linux 8 (Core)", 2020-09-12T13:58:08.617 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_version": "8", 2020-09-12T13:58:08.617 INFO:tasks.workunit.client.0.smithi027.stdout: "front_addr": "[v2:127.0.0.1:6810/27933,v1:127.0.0.1:6811/27933]", 2020-09-12T13:58:08.617 INFO:tasks.workunit.client.0.smithi027.stdout: "front_iface": "", 2020-09-12T13:58:08.618 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/27933,v1:127.0.0.1:6817/27933]", 2020-09-12T13:58:08.618 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/27933,v1:127.0.0.1:6815/27933]", 2020-09-12T13:58:08.618 INFO:tasks.workunit.client.0.smithi027.stdout: "hostname": "smithi027", 2020-09-12T13:58:08.618 INFO:tasks.workunit.client.0.smithi027.stdout: "journal_rotational": "1", 2020-09-12T13:58:08.618 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_description": "#1 SMP Sun Jul 26 03:54:29 UTC 2020", 2020-09-12T13:58:08.618 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_version": "4.18.0-193.14.2.el8_2.x86_64", 2020-09-12T13:58:08.618 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_swap_kb": "0", 2020-09-12T13:58:08.619 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_total_kb": "32659336", 2020-09-12T13:58:08.619 INFO:tasks.workunit.client.0.smithi027.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2020-09-12T13:58:08.619 INFO:tasks.workunit.client.0.smithi027.stdout: "objectstore_numa_unknown_devices": "sda", 2020-09-12T13:58:08.619 INFO:tasks.workunit.client.0.smithi027.stdout: "os": "Linux", 2020-09-12T13:58:08.619 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_data": "td/divergent-priors/2", 2020-09-12T13:58:08.619 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_objectstore": "bluestore", 2020-09-12T13:58:08.619 INFO:tasks.workunit.client.0.smithi027.stdout: "osdspec_affinity": "", 2020-09-12T13:58:08.619 INFO:tasks.workunit.client.0.smithi027.stdout: "rotational": "1" 2020-09-12T13:58:08.620 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.620 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T13:58:08.620 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap_clean_epochs": { 2020-09-12T13:58:08.620 INFO:tasks.workunit.client.0.smithi027.stdout: "min_last_epoch_clean": 0, 2020-09-12T13:58:08.620 INFO:tasks.workunit.client.0.smithi027.stdout: "last_epoch_clean": { 2020-09-12T13:58:08.620 INFO:tasks.workunit.client.0.smithi027.stdout: "per_pool": [] 2020-09-12T13:58:08.620 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.620 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_epochs": [ 2020-09-12T13:58:08.621 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.621 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T13:58:08.621 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 35 2020-09-12T13:58:08.621 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.621 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.621 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T13:58:08.621 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 10 2020-09-12T13:58:08.621 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.622 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.622 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T13:58:08.622 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 36 2020-09-12T13:58:08.622 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.622 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.622 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.622 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap_first_committed": 1, 2020-09-12T13:58:08.623 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap_last_committed": 41, 2020-09-12T13:58:08.623 INFO:tasks.workunit.client.0.smithi027.stdout: "crushmap": { 2020-09-12T13:58:08.623 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": [ 2020-09-12T13:58:08.623 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.623 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T13:58:08.623 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd.0", 2020-09-12T13:58:08.623 INFO:tasks.workunit.client.0.smithi027.stdout: "class": "hdd" 2020-09-12T13:58:08.623 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.624 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.624 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T13:58:08.624 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd.1", 2020-09-12T13:58:08.624 INFO:tasks.workunit.client.0.smithi027.stdout: "class": "hdd" 2020-09-12T13:58:08.624 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.624 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.624 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T13:58:08.624 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd.2", 2020-09-12T13:58:08.625 INFO:tasks.workunit.client.0.smithi027.stdout: "class": "hdd" 2020-09-12T13:58:08.625 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.625 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T13:58:08.625 INFO:tasks.workunit.client.0.smithi027.stdout: "types": [ 2020-09-12T13:58:08.625 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.625 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 0, 2020-09-12T13:58:08.625 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd" 2020-09-12T13:58:08.625 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.626 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.626 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 1, 2020-09-12T13:58:08.626 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "host" 2020-09-12T13:58:08.626 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.626 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.626 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 2, 2020-09-12T13:58:08.626 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "chassis" 2020-09-12T13:58:08.627 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.627 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.627 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 3, 2020-09-12T13:58:08.627 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "rack" 2020-09-12T13:58:08.627 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.627 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.627 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 4, 2020-09-12T13:58:08.627 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "row" 2020-09-12T13:58:08.628 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.628 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.628 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 5, 2020-09-12T13:58:08.628 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "pdu" 2020-09-12T13:58:08.628 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.628 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.628 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 6, 2020-09-12T13:58:08.628 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "pod" 2020-09-12T13:58:08.629 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.629 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.629 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 7, 2020-09-12T13:58:08.629 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "room" 2020-09-12T13:58:08.629 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.629 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.629 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 8, 2020-09-12T13:58:08.629 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "datacenter" 2020-09-12T13:58:08.630 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.630 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.630 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 9, 2020-09-12T13:58:08.630 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "zone" 2020-09-12T13:58:08.630 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.630 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.630 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 10, 2020-09-12T13:58:08.630 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "region" 2020-09-12T13:58:08.631 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.631 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.631 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 11, 2020-09-12T13:58:08.631 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "root" 2020-09-12T13:58:08.631 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.631 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T13:58:08.631 INFO:tasks.workunit.client.0.smithi027.stdout: "buckets": [ 2020-09-12T13:58:08.631 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.632 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -1, 2020-09-12T13:58:08.632 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "default", 2020-09-12T13:58:08.632 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 11, 2020-09-12T13:58:08.632 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "root", 2020-09-12T13:58:08.632 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T13:58:08.632 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T13:58:08.632 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T13:58:08.633 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T13:58:08.633 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.633 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -3, 2020-09-12T13:58:08.633 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T13:58:08.633 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T13:58:08.633 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.633 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.633 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.634 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.634 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -2, 2020-09-12T13:58:08.634 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "default~hdd", 2020-09-12T13:58:08.634 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 11, 2020-09-12T13:58:08.634 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "root", 2020-09-12T13:58:08.634 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T13:58:08.634 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T13:58:08.634 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T13:58:08.635 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T13:58:08.635 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.635 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -4, 2020-09-12T13:58:08.635 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T13:58:08.635 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T13:58:08.635 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.635 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.635 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.636 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.636 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -3, 2020-09-12T13:58:08.636 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "smithi027", 2020-09-12T13:58:08.636 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 1, 2020-09-12T13:58:08.636 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "host", 2020-09-12T13:58:08.636 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T13:58:08.636 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T13:58:08.636 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T13:58:08.637 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T13:58:08.637 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.637 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T13:58:08.637 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T13:58:08.637 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T13:58:08.637 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.637 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.638 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T13:58:08.638 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T13:58:08.638 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 1 2020-09-12T13:58:08.638 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.638 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.638 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T13:58:08.638 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T13:58:08.638 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 2 2020-09-12T13:58:08.639 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.639 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.639 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.639 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.639 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -4, 2020-09-12T13:58:08.639 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "smithi027~hdd", 2020-09-12T13:58:08.639 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 1, 2020-09-12T13:58:08.639 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "host", 2020-09-12T13:58:08.640 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T13:58:08.640 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T13:58:08.640 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T13:58:08.640 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T13:58:08.640 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.640 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T13:58:08.640 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T13:58:08.640 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T13:58:08.641 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.641 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.641 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T13:58:08.641 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T13:58:08.641 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 1 2020-09-12T13:58:08.641 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.641 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.641 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T13:58:08.642 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T13:58:08.642 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 2 2020-09-12T13:58:08.642 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.642 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.642 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.642 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T13:58:08.642 INFO:tasks.workunit.client.0.smithi027.stdout: "rules": [ 2020-09-12T13:58:08.643 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.643 INFO:tasks.workunit.client.0.smithi027.stdout: "rule_id": 0, 2020-09-12T13:58:08.643 INFO:tasks.workunit.client.0.smithi027.stdout: "rule_name": "replicated_rule", 2020-09-12T13:58:08.643 INFO:tasks.workunit.client.0.smithi027.stdout: "ruleset": 0, 2020-09-12T13:58:08.643 INFO:tasks.workunit.client.0.smithi027.stdout: "type": 1, 2020-09-12T13:58:08.643 INFO:tasks.workunit.client.0.smithi027.stdout: "min_size": 1, 2020-09-12T13:58:08.643 INFO:tasks.workunit.client.0.smithi027.stdout: "max_size": 10, 2020-09-12T13:58:08.643 INFO:tasks.workunit.client.0.smithi027.stdout: "steps": [ 2020-09-12T13:58:08.644 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.644 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "take", 2020-09-12T13:58:08.644 INFO:tasks.workunit.client.0.smithi027.stdout: "item": -1, 2020-09-12T13:58:08.644 INFO:tasks.workunit.client.0.smithi027.stdout: "item_name": "default" 2020-09-12T13:58:08.644 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.644 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.644 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "choose_firstn", 2020-09-12T13:58:08.644 INFO:tasks.workunit.client.0.smithi027.stdout: "num": 0, 2020-09-12T13:58:08.645 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "osd" 2020-09-12T13:58:08.645 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.645 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.645 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "emit" 2020-09-12T13:58:08.645 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.645 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T13:58:08.645 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.645 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T13:58:08.646 INFO:tasks.workunit.client.0.smithi027.stdout: "tunables": { 2020-09-12T13:58:08.646 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_local_tries": 0, 2020-09-12T13:58:08.646 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_local_fallback_tries": 0, 2020-09-12T13:58:08.646 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_total_tries": 50, 2020-09-12T13:58:08.646 INFO:tasks.workunit.client.0.smithi027.stdout: "chooseleaf_descend_once": 1, 2020-09-12T13:58:08.646 INFO:tasks.workunit.client.0.smithi027.stdout: "chooseleaf_vary_r": 1, 2020-09-12T13:58:08.646 INFO:tasks.workunit.client.0.smithi027.stdout: "chooseleaf_stable": 1, 2020-09-12T13:58:08.646 INFO:tasks.workunit.client.0.smithi027.stdout: "straw_calc_version": 1, 2020-09-12T13:58:08.647 INFO:tasks.workunit.client.0.smithi027.stdout: "allowed_bucket_algs": 54, 2020-09-12T13:58:08.647 INFO:tasks.workunit.client.0.smithi027.stdout: "profile": "jewel", 2020-09-12T13:58:08.647 INFO:tasks.workunit.client.0.smithi027.stdout: "optimal_tunables": 1, 2020-09-12T13:58:08.647 INFO:tasks.workunit.client.0.smithi027.stdout: "legacy_tunables": 0, 2020-09-12T13:58:08.647 INFO:tasks.workunit.client.0.smithi027.stdout: "minimum_required_version": "jewel", 2020-09-12T13:58:08.647 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables": 1, 2020-09-12T13:58:08.647 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables2": 1, 2020-09-12T13:58:08.647 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v2_rules": 0, 2020-09-12T13:58:08.648 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables3": 1, 2020-09-12T13:58:08.648 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v3_rules": 0, 2020-09-12T13:58:08.648 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v4_buckets": 1, 2020-09-12T13:58:08.648 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables5": 1, 2020-09-12T13:58:08.648 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v5_rules": 0 2020-09-12T13:58:08.648 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.648 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_args": {} 2020-09-12T13:58:08.649 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.649 INFO:tasks.workunit.client.0.smithi027.stdout: "fsmap": { 2020-09-12T13:58:08.649 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 1, 2020-09-12T13:58:08.649 INFO:tasks.workunit.client.0.smithi027.stdout: "default_fscid": -1, 2020-09-12T13:58:08.649 INFO:tasks.workunit.client.0.smithi027.stdout: "compat": { 2020-09-12T13:58:08.649 INFO:tasks.workunit.client.0.smithi027.stdout: "compat": {}, 2020-09-12T13:58:08.649 INFO:tasks.workunit.client.0.smithi027.stdout: "ro_compat": {}, 2020-09-12T13:58:08.649 INFO:tasks.workunit.client.0.smithi027.stdout: "incompat": { 2020-09-12T13:58:08.650 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_1": "base v0.20", 2020-09-12T13:58:08.650 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_2": "client writeable ranges", 2020-09-12T13:58:08.650 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_3": "default file layouts on dirs", 2020-09-12T13:58:08.650 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_4": "dir inode in separate object", 2020-09-12T13:58:08.650 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_5": "mds uses versioned encoding", 2020-09-12T13:58:08.650 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_6": "dirfrag is stored in omap", 2020-09-12T13:58:08.650 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_8": "no anchor table", 2020-09-12T13:58:08.650 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_9": "file layout v2", 2020-09-12T13:58:08.651 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_10": "snaprealm v2" 2020-09-12T13:58:08.651 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.651 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.651 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_flags": { 2020-09-12T13:58:08.651 INFO:tasks.workunit.client.0.smithi027.stdout: "enable_multiple": false, 2020-09-12T13:58:08.651 INFO:tasks.workunit.client.0.smithi027.stdout: "ever_enabled_multiple": false 2020-09-12T13:58:08.651 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.651 INFO:tasks.workunit.client.0.smithi027.stdout: "standbys": [], 2020-09-12T13:58:08.652 INFO:tasks.workunit.client.0.smithi027.stdout: "filesystems": [] 2020-09-12T13:58:08.652 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.652 INFO:tasks.workunit.client.0.smithi027.stdout: "mdsmap_first_committed": 1, 2020-09-12T13:58:08.652 INFO:tasks.workunit.client.0.smithi027.stdout: "mdsmap_last_committed": 1, 2020-09-12T13:58:08.652 INFO:tasks.workunit.client.0.smithi027.stdout: "auth": { 2020-09-12T13:58:08.652 INFO:tasks.workunit.client.0.smithi027.stdout: "first_committed": 1, 2020-09-12T13:58:08.652 INFO:tasks.workunit.client.0.smithi027.stdout: "last_committed": 5, 2020-09-12T13:58:08.652 INFO:tasks.workunit.client.0.smithi027.stdout: "num_secrets": 3 2020-09-12T13:58:08.653 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.653 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg": 1, 2020-09-12T13:58:08.653 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_active": 1, 2020-09-12T13:58:08.653 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_unknown": 0, 2020-09-12T13:58:08.653 INFO:tasks.workunit.client.0.smithi027.stdout: "num_osd": 3, 2020-09-12T13:58:08.653 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_sum": { 2020-09-12T13:58:08.653 INFO:tasks.workunit.client.0.smithi027.stdout: "stat_sum": { 2020-09-12T13:58:08.653 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes": 48300, 2020-09-12T13:58:08.654 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects": 100, 2020-09-12T13:58:08.654 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_clones": 0, 2020-09-12T13:58:08.654 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_copies": 300, 2020-09-12T13:58:08.654 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing_on_primary": 0, 2020-09-12T13:58:08.654 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing": 0, 2020-09-12T13:58:08.654 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_degraded": 100, 2020-09-12T13:58:08.654 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_misplaced": 0, 2020-09-12T13:58:08.655 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_unfound": 0, 2020-09-12T13:58:08.655 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_dirty": 100, 2020-09-12T13:58:08.655 INFO:tasks.workunit.client.0.smithi027.stdout: "num_whiteouts": 0, 2020-09-12T13:58:08.655 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read": 0, 2020-09-12T13:58:08.655 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read_kb": 0, 2020-09-12T13:58:08.655 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write": 100, 2020-09-12T13:58:08.655 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write_kb": 100, 2020-09-12T13:58:08.655 INFO:tasks.workunit.client.0.smithi027.stdout: "num_scrub_errors": 0, 2020-09-12T13:58:08.656 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shallow_scrub_errors": 0, 2020-09-12T13:58:08.656 INFO:tasks.workunit.client.0.smithi027.stdout: "num_deep_scrub_errors": 0, 2020-09-12T13:58:08.656 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_recovered": 0, 2020-09-12T13:58:08.656 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_recovered": 0, 2020-09-12T13:58:08.656 INFO:tasks.workunit.client.0.smithi027.stdout: "num_keys_recovered": 0, 2020-09-12T13:58:08.656 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_omap": 0, 2020-09-12T13:58:08.656 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_hit_set_archive": 0, 2020-09-12T13:58:08.656 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_hit_set_archive": 0, 2020-09-12T13:58:08.657 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush": 0, 2020-09-12T13:58:08.657 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_kb": 0, 2020-09-12T13:58:08.657 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict": 0, 2020-09-12T13:58:08.657 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_kb": 0, 2020-09-12T13:58:08.657 INFO:tasks.workunit.client.0.smithi027.stdout: "num_promote": 0, 2020-09-12T13:58:08.657 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_high": 0, 2020-09-12T13:58:08.657 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_low": 0, 2020-09-12T13:58:08.657 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_some": 0, 2020-09-12T13:58:08.658 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_full": 0, 2020-09-12T13:58:08.658 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_pinned": 0, 2020-09-12T13:58:08.658 INFO:tasks.workunit.client.0.smithi027.stdout: "num_legacy_snapsets": 0, 2020-09-12T13:58:08.658 INFO:tasks.workunit.client.0.smithi027.stdout: "num_large_omap_objects": 0, 2020-09-12T13:58:08.658 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_manifest": 0, 2020-09-12T13:58:08.658 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_bytes": 0, 2020-09-12T13:58:08.658 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_keys": 0, 2020-09-12T13:58:08.658 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_repaired": 0 2020-09-12T13:58:08.659 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.659 INFO:tasks.workunit.client.0.smithi027.stdout: "store_stats": { 2020-09-12T13:58:08.659 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 0, 2020-09-12T13:58:08.659 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 0, 2020-09-12T13:58:08.659 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T13:58:08.659 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 0, 2020-09-12T13:58:08.659 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 0, 2020-09-12T13:58:08.660 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T13:58:08.660 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T13:58:08.660 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T13:58:08.660 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 0, 2020-09-12T13:58:08.660 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 0 2020-09-12T13:58:08.661 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.661 INFO:tasks.workunit.client.0.smithi027.stdout: "log_size": 16, 2020-09-12T13:58:08.661 INFO:tasks.workunit.client.0.smithi027.stdout: "ondisk_log_size": 16, 2020-09-12T13:58:08.661 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 2, 2020-09-12T13:58:08.661 INFO:tasks.workunit.client.0.smithi027.stdout: "acting": 2, 2020-09-12T13:58:08.661 INFO:tasks.workunit.client.0.smithi027.stdout: "num_store_stats": 0 2020-09-12T13:58:08.661 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.661 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_sum": { 2020-09-12T13:58:08.662 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 0, 2020-09-12T13:58:08.662 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 0, 2020-09-12T13:58:08.662 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pgs": 3, 2020-09-12T13:58:08.662 INFO:tasks.workunit.client.0.smithi027.stdout: "num_osds": 3, 2020-09-12T13:58:08.662 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_osds": 3, 2020-09-12T13:58:08.662 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_omap_osds": 3, 2020-09-12T13:58:08.662 INFO:tasks.workunit.client.0.smithi027.stdout: "kb": 314572800, 2020-09-12T13:58:08.662 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used": 18332, 2020-09-12T13:58:08.663 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_data": 2112, 2020-09-12T13:58:08.663 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_omap": 1, 2020-09-12T13:58:08.663 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_meta": 16190, 2020-09-12T13:58:08.663 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_avail": 314554468, 2020-09-12T13:58:08.663 INFO:tasks.workunit.client.0.smithi027.stdout: "statfs": { 2020-09-12T13:58:08.663 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 322122547200, 2020-09-12T13:58:08.663 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 322103775232, 2020-09-12T13:58:08.663 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T13:58:08.664 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 2162688, 2020-09-12T13:58:08.664 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 415894, 2020-09-12T13:58:08.664 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T13:58:08.664 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T13:58:08.664 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T13:58:08.664 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 1928, 2020-09-12T13:58:08.664 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 16578680 2020-09-12T13:58:08.665 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.665 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_peers": [], 2020-09-12T13:58:08.665 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_trim_queue_len": 0, 2020-09-12T13:58:08.665 INFO:tasks.workunit.client.0.smithi027.stdout: "num_snap_trimming": 0, 2020-09-12T13:58:08.665 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shards_repaired": 1, 2020-09-12T13:58:08.665 INFO:tasks.workunit.client.0.smithi027.stdout: "op_queue_age_hist": { 2020-09-12T13:58:08.665 INFO:tasks.workunit.client.0.smithi027.stdout: "histogram": [], 2020-09-12T13:58:08.665 INFO:tasks.workunit.client.0.smithi027.stdout: "upper_bound": 1 2020-09-12T13:58:08.666 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.666 INFO:tasks.workunit.client.0.smithi027.stdout: "perf_stat": { 2020-09-12T13:58:08.666 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ms": 0, 2020-09-12T13:58:08.666 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ms": 0, 2020-09-12T13:58:08.666 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ns": 0, 2020-09-12T13:58:08.666 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ns": 0 2020-09-12T13:58:08.666 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.666 INFO:tasks.workunit.client.0.smithi027.stdout: "alerts": [], 2020-09-12T13:58:08.667 INFO:tasks.workunit.client.0.smithi027.stdout: "network_ping_times": [] 2020-09-12T13:58:08.667 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.667 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_sum_by_class": { 2020-09-12T13:58:08.667 INFO:tasks.workunit.client.0.smithi027.stdout: "hdd": { 2020-09-12T13:58:08.667 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 0, 2020-09-12T13:58:08.667 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 0, 2020-09-12T13:58:08.667 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pgs": 3, 2020-09-12T13:58:08.667 INFO:tasks.workunit.client.0.smithi027.stdout: "num_osds": 3, 2020-09-12T13:58:08.668 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_osds": 3, 2020-09-12T13:58:08.668 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_omap_osds": 3, 2020-09-12T13:58:08.668 INFO:tasks.workunit.client.0.smithi027.stdout: "kb": 314572800, 2020-09-12T13:58:08.668 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used": 18332, 2020-09-12T13:58:08.668 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_data": 2112, 2020-09-12T13:58:08.668 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_omap": 1, 2020-09-12T13:58:08.668 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_meta": 16190, 2020-09-12T13:58:08.668 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_avail": 314554468, 2020-09-12T13:58:08.669 INFO:tasks.workunit.client.0.smithi027.stdout: "statfs": { 2020-09-12T13:58:08.669 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 322122547200, 2020-09-12T13:58:08.669 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 322103775232, 2020-09-12T13:58:08.669 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T13:58:08.669 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 2162688, 2020-09-12T13:58:08.669 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 415894, 2020-09-12T13:58:08.669 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T13:58:08.670 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T13:58:08.670 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T13:58:08.670 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 1928, 2020-09-12T13:58:08.670 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 16578680 2020-09-12T13:58:08.670 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.670 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_peers": [], 2020-09-12T13:58:08.670 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_trim_queue_len": 0, 2020-09-12T13:58:08.670 INFO:tasks.workunit.client.0.smithi027.stdout: "num_snap_trimming": 0, 2020-09-12T13:58:08.671 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shards_repaired": 1, 2020-09-12T13:58:08.671 INFO:tasks.workunit.client.0.smithi027.stdout: "op_queue_age_hist": { 2020-09-12T13:58:08.671 INFO:tasks.workunit.client.0.smithi027.stdout: "histogram": [], 2020-09-12T13:58:08.671 INFO:tasks.workunit.client.0.smithi027.stdout: "upper_bound": 1 2020-09-12T13:58:08.671 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.671 INFO:tasks.workunit.client.0.smithi027.stdout: "perf_stat": { 2020-09-12T13:58:08.671 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ms": 0, 2020-09-12T13:58:08.671 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ms": 0, 2020-09-12T13:58:08.672 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ns": 0, 2020-09-12T13:58:08.672 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ns": 0 2020-09-12T13:58:08.672 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.672 INFO:tasks.workunit.client.0.smithi027.stdout: "alerts": [], 2020-09-12T13:58:08.672 INFO:tasks.workunit.client.0.smithi027.stdout: "network_ping_times": [] 2020-09-12T13:58:08.672 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.672 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.672 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_stats": [ 2020-09-12T13:58:08.673 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.673 INFO:tasks.workunit.client.0.smithi027.stdout: "poolid": 1, 2020-09-12T13:58:08.673 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg": 1, 2020-09-12T13:58:08.673 INFO:tasks.workunit.client.0.smithi027.stdout: "stat_sum": { 2020-09-12T13:58:08.673 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes": 48300, 2020-09-12T13:58:08.673 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects": 100, 2020-09-12T13:58:08.673 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_clones": 0, 2020-09-12T13:58:08.673 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_copies": 300, 2020-09-12T13:58:08.674 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing_on_primary": 0, 2020-09-12T13:58:08.674 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing": 0, 2020-09-12T13:58:08.674 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_degraded": 100, 2020-09-12T13:58:08.674 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_misplaced": 0, 2020-09-12T13:58:08.674 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_unfound": 0, 2020-09-12T13:58:08.674 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_dirty": 100, 2020-09-12T13:58:08.674 INFO:tasks.workunit.client.0.smithi027.stdout: "num_whiteouts": 0, 2020-09-12T13:58:08.675 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read": 0, 2020-09-12T13:58:08.675 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read_kb": 0, 2020-09-12T13:58:08.675 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write": 100, 2020-09-12T13:58:08.675 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write_kb": 100, 2020-09-12T13:58:08.675 INFO:tasks.workunit.client.0.smithi027.stdout: "num_scrub_errors": 0, 2020-09-12T13:58:08.675 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shallow_scrub_errors": 0, 2020-09-12T13:58:08.675 INFO:tasks.workunit.client.0.smithi027.stdout: "num_deep_scrub_errors": 0, 2020-09-12T13:58:08.675 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_recovered": 0, 2020-09-12T13:58:08.676 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_recovered": 0, 2020-09-12T13:58:08.676 INFO:tasks.workunit.client.0.smithi027.stdout: "num_keys_recovered": 0, 2020-09-12T13:58:08.676 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_omap": 0, 2020-09-12T13:58:08.676 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_hit_set_archive": 0, 2020-09-12T13:58:08.676 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_hit_set_archive": 0, 2020-09-12T13:58:08.676 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush": 0, 2020-09-12T13:58:08.676 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_kb": 0, 2020-09-12T13:58:08.676 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict": 0, 2020-09-12T13:58:08.677 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_kb": 0, 2020-09-12T13:58:08.677 INFO:tasks.workunit.client.0.smithi027.stdout: "num_promote": 0, 2020-09-12T13:58:08.677 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_high": 0, 2020-09-12T13:58:08.677 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_low": 0, 2020-09-12T13:58:08.677 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_some": 0, 2020-09-12T13:58:08.677 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_full": 0, 2020-09-12T13:58:08.677 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_pinned": 0, 2020-09-12T13:58:08.677 INFO:tasks.workunit.client.0.smithi027.stdout: "num_legacy_snapsets": 0, 2020-09-12T13:58:08.678 INFO:tasks.workunit.client.0.smithi027.stdout: "num_large_omap_objects": 0, 2020-09-12T13:58:08.678 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_manifest": 0, 2020-09-12T13:58:08.678 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_bytes": 0, 2020-09-12T13:58:08.678 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_keys": 0, 2020-09-12T13:58:08.678 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_repaired": 0 2020-09-12T13:58:08.678 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.678 INFO:tasks.workunit.client.0.smithi027.stdout: "store_stats": { 2020-09-12T13:58:08.678 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 0, 2020-09-12T13:58:08.679 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 0, 2020-09-12T13:58:08.679 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T13:58:08.679 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 1228800, 2020-09-12T13:58:08.679 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 144900, 2020-09-12T13:58:08.679 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T13:58:08.679 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T13:58:08.679 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T13:58:08.680 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 0, 2020-09-12T13:58:08.680 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 0 2020-09-12T13:58:08.680 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.680 INFO:tasks.workunit.client.0.smithi027.stdout: "log_size": 16, 2020-09-12T13:58:08.680 INFO:tasks.workunit.client.0.smithi027.stdout: "ondisk_log_size": 16, 2020-09-12T13:58:08.680 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 2, 2020-09-12T13:58:08.680 INFO:tasks.workunit.client.0.smithi027.stdout: "acting": 2, 2020-09-12T13:58:08.680 INFO:tasks.workunit.client.0.smithi027.stdout: "num_store_stats": 3 2020-09-12T13:58:08.681 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.681 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T13:58:08.681 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_stats": [ 2020-09-12T13:58:08.681 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.681 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T13:58:08.681 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 150323855371 2020-09-12T13:58:08.681 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.681 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.682 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T13:58:08.682 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 42949672977 2020-09-12T13:58:08.682 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.682 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.682 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T13:58:08.682 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 154618822666 2020-09-12T13:58:08.682 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.682 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T13:58:08.683 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_by_state": [ 2020-09-12T13:58:08.683 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.683 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "active+undersized+degraded", 2020-09-12T13:58:08.683 INFO:tasks.workunit.client.0.smithi027.stdout: "num": 1 2020-09-12T13:58:08.683 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.683 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T13:58:08.683 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_by_osd": [ 2020-09-12T13:58:08.683 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.684 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T13:58:08.684 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 0, 2020-09-12T13:58:08.684 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 1, 2020-09-12T13:58:08.684 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T13:58:08.684 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.684 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.684 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T13:58:08.684 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 0, 2020-09-12T13:58:08.685 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 0, 2020-09-12T13:58:08.685 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T13:58:08.685 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.685 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.685 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T13:58:08.685 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 1, 2020-09-12T13:58:08.685 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 1, 2020-09-12T13:58:08.686 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T13:58:08.686 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.686 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T13:58:08.686 INFO:tasks.workunit.client.0.smithi027.stdout: "purged_snaps": [ 2020-09-12T13:58:08.686 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T13:58:08.686 INFO:tasks.workunit.client.0.smithi027.stdout: "pool": 1, 2020-09-12T13:58:08.686 INFO:tasks.workunit.client.0.smithi027.stdout: "purged_snaps": {} 2020-09-12T13:58:08.686 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.687 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T13:58:08.687 INFO:tasks.workunit.client.0.smithi027.stdout: "servicemap": { 2020-09-12T13:58:08.687 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 4, 2020-09-12T13:58:08.687 INFO:tasks.workunit.client.0.smithi027.stdout: "modified": "2020-09-12T13:57:29.526606+0000", 2020-09-12T13:58:08.687 INFO:tasks.workunit.client.0.smithi027.stdout: "services": { 2020-09-12T13:58:08.687 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": { 2020-09-12T13:58:08.687 INFO:tasks.workunit.client.0.smithi027.stdout: "daemons": { 2020-09-12T13:58:08.687 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": "", 2020-09-12T13:58:08.688 INFO:tasks.workunit.client.0.smithi027.stdout: "0": { 2020-09-12T13:58:08.688 INFO:tasks.workunit.client.0.smithi027.stdout: "start_epoch": 0, 2020-09-12T13:58:08.688 INFO:tasks.workunit.client.0.smithi027.stdout: "start_stamp": "0.000000", 2020-09-12T13:58:08.688 INFO:tasks.workunit.client.0.smithi027.stdout: "gid": 0, 2020-09-12T13:58:08.688 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "(unrecognized address family 0)/0", 2020-09-12T13:58:08.688 INFO:tasks.workunit.client.0.smithi027.stdout: "metadata": {}, 2020-09-12T13:58:08.688 INFO:tasks.workunit.client.0.smithi027.stdout: "task_status": {} 2020-09-12T13:58:08.689 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.689 INFO:tasks.workunit.client.0.smithi027.stdout: "2": { 2020-09-12T13:58:08.689 INFO:tasks.workunit.client.0.smithi027.stdout: "start_epoch": 0, 2020-09-12T13:58:08.690 INFO:tasks.workunit.client.0.smithi027.stdout: "start_stamp": "0.000000", 2020-09-12T13:58:08.690 INFO:tasks.workunit.client.0.smithi027.stdout: "gid": 0, 2020-09-12T13:58:08.690 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "(unrecognized address family 0)/0", 2020-09-12T13:58:08.690 INFO:tasks.workunit.client.0.smithi027.stdout: "metadata": {}, 2020-09-12T13:58:08.690 INFO:tasks.workunit.client.0.smithi027.stdout: "task_status": {} 2020-09-12T13:58:08.690 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.690 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.690 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.691 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.691 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T13:58:08.691 INFO:tasks.workunit.client.0.smithi027.stdout: "paxos": { 2020-09-12T13:58:08.691 INFO:tasks.workunit.client.0.smithi027.stdout: "first_committed": 1, 2020-09-12T13:58:08.691 INFO:tasks.workunit.client.0.smithi027.stdout: "last_committed": 357, 2020-09-12T13:58:08.691 INFO:tasks.workunit.client.0.smithi027.stdout: "last_pn": 0, 2020-09-12T13:58:08.691 INFO:tasks.workunit.client.0.smithi027.stdout: "accepted_pn": 0 2020-09-12T13:58:08.691 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T13:58:08.692 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T13:58:08.692 INFO:tasks.workunit.client.0.smithi027.stderr:report 2003421966 2020-09-12T13:58:08.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1604: wait_for_clean: return 1 2020-09-12T13:58:08.693 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:153: TEST_divergent: expr 2 + 2 2020-09-12T13:58:08.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:153: TEST_divergent: objname=existing_4 2020-09-12T13:58:08.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:154: TEST_divergent: echo 'writing non-divergent object existing_4' 2020-09-12T13:58:08.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:155: TEST_divergent: ceph pg dump pgs 2020-09-12T13:58:08.693 INFO:tasks.workunit.client.0.smithi027.stdout:writing non-divergent object existing_4 2020-09-12T13:58:08.933 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T13:58:08.933 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 0 100 0 0 48300 0 0 16 16 active+undersized+degraded 2020-09-12T13:57:31.745892+0000 29'100 41:123 [0,2] 0 [0,2] 0 0'0 2020-09-12T13:56:39.394566+0000 0'0 2020-09-12T13:56:39.394566+0000 0 2020-09-12T13:58:08.933 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T13:58:08.934 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T13:58:08.935 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T13:58:08.944 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:156: TEST_divergent: rados -p test put existing_4 /etc/resolv.conf 2020-09-12T13:58:09.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:159: TEST_divergent: echo 'delay recovery' 2020-09-12T13:58:09.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:160: TEST_divergent: ceph pg dump pgs 2020-09-12T13:58:09.076 INFO:tasks.workunit.client.0.smithi027.stdout:delay recovery 2020-09-12T13:58:09.459 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T13:58:09.459 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 0 100 0 0 48300 0 0 16 16 active+undersized+degraded 2020-09-12T13:57:31.745892+0000 29'100 41:123 [0,2] 0 [0,2] 0 0'0 2020-09-12T13:56:39.394566+0000 0'0 2020-09-12T13:56:39.394566+0000 0 2020-09-12T13:58:09.459 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T13:58:09.459 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T13:58:09.460 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T13:58:09.470 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:161: TEST_divergent: for i in $non_divergent 2020-09-12T13:58:09.471 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: get_asok_path osd.0 2020-09-12T13:58:09.471 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2020-09-12T13:58:09.471 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2020-09-12T13:58:09.471 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T13:58:09.471 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:58:09.472 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:58:09.472 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.0.asok 2020-09-12T13:58:09.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: CEPH_ARGS= 2020-09-12T13:58:09.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.0.asok set_recovery_delay 100000 2020-09-12T13:58:09.578 INFO:tasks.workunit.client.0.smithi027.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2020-09-12T13:58:09.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:161: TEST_divergent: for i in $non_divergent 2020-09-12T13:58:09.587 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: get_asok_path osd.2 2020-09-12T13:58:09.588 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2020-09-12T13:58:09.588 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2020-09-12T13:58:09.588 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T13:58:09.588 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:58:09.588 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:58:09.588 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.2.asok 2020-09-12T13:58:09.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: CEPH_ARGS= 2020-09-12T13:58:09.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.2.asok set_recovery_delay 100000 2020-09-12T13:58:09.695 INFO:tasks.workunit.client.0.smithi027.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2020-09-12T13:58:09.703 INFO:tasks.workunit.client.0.smithi027.stdout:revive divergent 1 2020-09-12T13:58:09.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:167: TEST_divergent: echo 'revive divergent 1' 2020-09-12T13:58:09.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:168: TEST_divergent: ceph pg dump pgs 2020-09-12T13:58:10.086 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T13:58:10.086 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 0 100 0 0 48300 0 0 16 16 active+undersized+degraded 2020-09-12T13:57:31.745892+0000 29'100 41:123 [0,2] 0 [0,2] 0 0'0 2020-09-12T13:56:39.394566+0000 0'0 2020-09-12T13:56:39.394566+0000 0 2020-09-12T13:58:10.087 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T13:58:10.087 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T13:58:10.088 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T13:58:10.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:169: TEST_divergent: ceph osd set noup 2020-09-12T13:58:10.788 INFO:tasks.workunit.client.0.smithi027.stderr:noup is set 2020-09-12T13:58:10.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:170: TEST_divergent: activate_osd td/divergent-priors 1 2020-09-12T13:58:10.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/divergent-priors 2020-09-12T13:58:10.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T13:58:10.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=1 2020-09-12T13:58:10.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T13:58:10.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/divergent-priors/1 2020-09-12T13:58:10.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T13:58:10.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T13:58:10.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T13:58:10.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T13:58:10.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2020-09-12T13:58:10.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2020-09-12T13:58:10.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T13:58:10.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T13:58:10.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T13:58:10.806 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T13:58:10.806 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T13:58:10.806 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T13:58:10.806 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T13:58:10.807 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:58:10.807 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:58:10.807 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:58:10.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:58:10.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T13:58:10.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T13:58:10.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T13:58:10.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T13:58:10.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T13:58:10.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:58:10.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T13:58:10.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T13:58:10.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/divergent-priors/1 2020-09-12T13:58:10.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.1 2020-09-12T13:58:10.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 1 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:58:10.811 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/divergent-priors/1/whoami 2020-09-12T13:58:10.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 1 = 1 ']' 2020-09-12T13:58:10.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T13:58:10.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T13:58:10.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T13:58:10.813 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T13:58:10.829 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:58:10.827+0000 7fee10f9aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:58:10.843 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:58:10.841+0000 7fee10f9aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:58:10.845 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:58:10.843+0000 7fee10f9aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:58:11.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:877: activate_osd: return 0 2020-09-12T13:58:11.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:171: TEST_divergent: sleep 5 2020-09-12T13:58:11.365 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:58:11.363+0000 7fee10f9aec0 -1 Falling back to public interface 2020-09-12T13:58:12.551 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:58:12.549+0000 7fee10f9aec0 -1 osd.1 29 log_to_monitors {default=true} 2020-09-12T13:58:16.211 INFO:tasks.workunit.client.0.smithi027.stdout:delay recovery divergent 2020-09-12T13:58:16.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:173: TEST_divergent: echo 'delay recovery divergent' 2020-09-12T13:58:16.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:174: TEST_divergent: ceph pg dump pgs 2020-09-12T13:58:16.604 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T13:58:16.605 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 0 100 0 0 47947 0 0 17 17 active+undersized+degraded 2020-09-12T13:57:31.745892+0000 41'101 42:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T13:56:39.394566+0000 0'0 2020-09-12T13:56:39.394566+0000 0 2020-09-12T13:58:16.605 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T13:58:16.605 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T13:58:16.606 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T13:58:16.616 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:175: TEST_divergent: get_asok_path osd.1 2020-09-12T13:58:16.616 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2020-09-12T13:58:16.616 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2020-09-12T13:58:16.616 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T13:58:16.617 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:58:16.617 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:58:16.617 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.1.asok 2020-09-12T13:58:16.617 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:175: TEST_divergent: CEPH_ARGS= 2020-09-12T13:58:16.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:175: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.1.asok set_recovery_delay 100000 2020-09-12T13:58:16.723 INFO:tasks.workunit.client.0.smithi027.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2020-09-12T13:58:16.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:177: TEST_divergent: ceph osd unset noup 2020-09-12T13:58:17.497 INFO:tasks.workunit.client.0.smithi027.stderr:noup is unset 2020-09-12T13:58:17.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:179: TEST_divergent: wait_for_osd up 0 2020-09-12T13:58:17.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T13:58:17.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T13:58:17.508 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T13:58:17.508 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T13:58:17.508 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:58:17.508 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T13:58:17.508 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:58:17.508 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T13:58:17.509 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T13:58:18.010 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 35 up_thru 39 down_at 30 last_clean_interval [5,29) [v2:127.0.0.1:6802/27534,v1:127.0.0.1:6803/27534] [v2:127.0.0.1:6804/27534,v1:127.0.0.1:6805/27534] exists,up 97c8d704-849c-44e1-a64d-c66c37f051bf 2020-09-12T13:58:18.011 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T13:58:18.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T13:58:18.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T13:58:18.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T13:58:18.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:180: TEST_divergent: wait_for_osd up 1 2020-09-12T13:58:18.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T13:58:18.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T13:58:18.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T13:58:18.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T13:58:18.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:58:18.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T13:58:18.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:58:18.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T13:58:18.461 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 45 up_thru 45 down_at 31 last_clean_interval [10,29) [v2:127.0.0.1:6818/29489,v1:127.0.0.1:6819/29489] [v2:127.0.0.1:6820/29489,v1:127.0.0.1:6821/29489] exists,up 96117818-e665-481f-9e92-b70e5d147a31 2020-09-12T13:58:18.462 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T13:58:18.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T13:58:18.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T13:58:18.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T13:58:18.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:181: TEST_divergent: wait_for_osd up 2 2020-09-12T13:58:18.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T13:58:18.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T13:58:18.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T13:58:18.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T13:58:18.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:58:18.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T13:58:18.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:58:18.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T13:58:18.866 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up in weight 1 up_from 36 up_thru 0 down_at 32 last_clean_interval [15,29) [v2:127.0.0.1:6810/27933,v1:127.0.0.1:6811/27933] [v2:127.0.0.1:6812/27933,v1:127.0.0.1:6813/27933] exists,up cadcbbd5-8cd1-4081-9e23-e263c65059d2 2020-09-12T13:58:18.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T13:58:18.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T13:58:18.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T13:58:18.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:183: TEST_divergent: ceph pg dump pgs 2020-09-12T13:58:19.255 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T13:58:19.256 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 0 100 0 0 47947 0 0 17 17 active+undersized+degraded 2020-09-12T13:57:31.745892+0000 41'101 42:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T13:56:39.394566+0000 0'0 2020-09-12T13:56:39.394566+0000 0 2020-09-12T13:58:19.256 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T13:58:19.256 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T13:58:19.257 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T13:58:19.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:184: TEST_divergent: echo 'wait for peering' 2020-09-12T13:58:19.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:185: TEST_divergent: ceph pg dump pgs 2020-09-12T13:58:19.267 INFO:tasks.workunit.client.0.smithi027.stdout:wait for peering 2020-09-12T13:58:19.647 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T13:58:19.647 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 0 100 0 0 47947 0 0 17 17 active+undersized+degraded 2020-09-12T13:57:31.745892+0000 41'101 42:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T13:56:39.394566+0000 0'0 2020-09-12T13:56:39.394566+0000 0 2020-09-12T13:58:19.647 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T13:58:19.648 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T13:58:19.649 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T13:58:19.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:186: TEST_divergent: rados -p test put foo /etc/fstab 2020-09-12T13:58:27.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:188: TEST_divergent: echo 'killing divergent 1' 2020-09-12T13:58:27.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:189: TEST_divergent: ceph pg dump pgs 2020-09-12T13:58:27.889 INFO:tasks.workunit.client.0.smithi027.stdout:killing divergent 1 2020-09-12T13:58:28.285 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T13:58:28.285 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 2 2 0 0 47947 0 0 17 17 active+recovering+degraded+wait 2020-09-12T13:58:18.441239+0000 41'101 46:132 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T13:56:39.394566+0000 0'0 2020-09-12T13:56:39.394566+0000 0 2020-09-12T13:58:28.285 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T13:58:28.285 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T13:58:28.287 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T13:58:28.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:190: TEST_divergent: kill_daemons td/divergent-priors KILL osd.1 2020-09-12T13:58:28.296 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T13:58:28.297 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T13:58:28.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T13:58:28.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T13:58:28.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T13:58:28.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T13:58:28.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:192: TEST_divergent: echo 'reviving divergent 1' 2020-09-12T13:58:28.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:193: TEST_divergent: ceph pg dump pgs 2020-09-12T13:58:28.403 INFO:tasks.workunit.client.0.smithi027.stdout:reviving divergent 1 2020-09-12T13:58:28.794 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T13:58:28.795 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 2 2 0 0 47947 0 0 17 17 active+recovering+degraded+wait 2020-09-12T13:58:18.441239+0000 41'101 46:132 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T13:56:39.394566+0000 0'0 2020-09-12T13:56:39.394566+0000 0 2020-09-12T13:58:28.795 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T13:58:28.795 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T13:58:28.796 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T13:58:28.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:194: TEST_divergent: activate_osd td/divergent-priors 1 2020-09-12T13:58:28.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/divergent-priors 2020-09-12T13:58:28.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T13:58:28.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=1 2020-09-12T13:58:28.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T13:58:28.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/divergent-priors/1 2020-09-12T13:58:28.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T13:58:28.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T13:58:28.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T13:58:28.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T13:58:28.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2020-09-12T13:58:28.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2020-09-12T13:58:28.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T13:58:28.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T13:58:28.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T13:58:28.809 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T13:58:28.810 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T13:58:28.810 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T13:58:28.810 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T13:58:28.810 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:58:28.810 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:58:28.810 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:58:28.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:58:28.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T13:58:28.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T13:58:28.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T13:58:28.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T13:58:28.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T13:58:28.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:58:28.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T13:58:28.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T13:58:28.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/divergent-priors/1 2020-09-12T13:58:28.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.1 2020-09-12T13:58:28.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 1 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:58:28.813 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/divergent-priors/1/whoami 2020-09-12T13:58:28.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 1 = 1 ']' 2020-09-12T13:58:28.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T13:58:28.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T13:58:28.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T13:58:28.814 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T13:58:28.837 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:58:28.836+0000 7fc56cedeec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:58:28.839 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:58:28.838+0000 7fc56cedeec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:58:28.840 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:58:28.840+0000 7fc56cedeec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:58:29.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: wait_for_osd up 1 2020-09-12T13:58:29.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T13:58:29.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T13:58:29.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T13:58:29.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T13:58:29.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:58:29.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T13:58:29.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:58:29.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T13:58:29.227 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T13:58:29.361 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:58:29.360+0000 7fc56cedeec0 -1 Falling back to public interface 2020-09-12T13:58:29.633 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 45 up_thru 45 down_at 31 last_clean_interval [10,29) [v2:127.0.0.1:6818/29489,v1:127.0.0.1:6819/29489] [v2:127.0.0.1:6820/29489,v1:127.0.0.1:6821/29489] exists,up 96117818-e665-481f-9e92-b70e5d147a31 2020-09-12T13:58:29.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T13:58:29.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T13:58:29.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T13:58:29.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:196: TEST_divergent: sleep 20 2020-09-12T13:58:30.453 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:58:30.452+0000 7fc56cedeec0 -1 osd.1 46 log_to_monitors {default=true} 2020-09-12T13:58:49.634 INFO:tasks.workunit.client.0.smithi027.stdout:allowing recovery 2020-09-12T13:58:49.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:198: TEST_divergent: echo 'allowing recovery' 2020-09-12T13:58:49.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:199: TEST_divergent: ceph pg dump pgs 2020-09-12T13:58:50.028 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T13:58:50.029 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 101 0 0 0 0 48430 0 0 18 18 active+clean 2020-09-12T13:58:31.915337+0000 46'102 50:150 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T13:56:39.394566+0000 0'0 2020-09-12T13:56:39.394566+0000 0 2020-09-12T13:58:50.029 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T13:58:50.029 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T13:58:50.030 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T13:58:50.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:201: TEST_divergent: for i in $osds 2020-09-12T13:58:50.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:203: TEST_divergent: ceph tell osd.0 debug kick_recovery_wq 0 2020-09-12T13:58:50.177 INFO:tasks.workunit.client.0.smithi027.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2020-09-12T13:58:50.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:201: TEST_divergent: for i in $osds 2020-09-12T13:58:50.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:203: TEST_divergent: ceph tell osd.1 debug kick_recovery_wq 0 2020-09-12T13:58:50.323 INFO:tasks.workunit.client.0.smithi027.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2020-09-12T13:58:50.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:201: TEST_divergent: for i in $osds 2020-09-12T13:58:50.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:203: TEST_divergent: ceph tell osd.2 debug kick_recovery_wq 0 2020-09-12T13:58:50.469 INFO:tasks.workunit.client.0.smithi027.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2020-09-12T13:58:50.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:206: TEST_divergent: echo 'reading divergent objects' 2020-09-12T13:58:50.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:207: TEST_divergent: ceph pg dump pgs 2020-09-12T13:58:50.478 INFO:tasks.workunit.client.0.smithi027.stdout:reading divergent objects 2020-09-12T13:58:50.864 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T13:58:50.864 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 101 0 0 0 0 48430 0 0 18 18 active+clean 2020-09-12T13:58:31.915337+0000 46'102 50:150 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T13:56:39.394566+0000 0'0 2020-09-12T13:56:39.394566+0000 0 2020-09-12T13:58:50.865 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T13:58:50.865 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T13:58:50.866 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T13:58:50.875 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: expr 2 + 2 2020-09-12T13:58:50.876 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: seq 1 4 2020-09-12T13:58:50.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2020-09-12T13:58:50.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:210: TEST_divergent: rados -p test get existing_1 td/divergent-priors/existing 2020-09-12T13:58:50.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2020-09-12T13:58:50.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:210: TEST_divergent: rados -p test get existing_2 td/divergent-priors/existing 2020-09-12T13:58:50.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2020-09-12T13:58:50.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:210: TEST_divergent: rados -p test get existing_3 td/divergent-priors/existing 2020-09-12T13:58:51.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2020-09-12T13:58:51.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:210: TEST_divergent: rados -p test get existing_4 td/divergent-priors/existing 2020-09-12T13:58:51.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:212: TEST_divergent: rm -f td/divergent-priors/existing 2020-09-12T13:58:51.044 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:214: TEST_divergent: find td/divergent-priors -name '*osd*log' 2020-09-12T13:58:51.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:214: TEST_divergent: grep _merge_object_divergent_entries td/divergent-priors/osd.2.log td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log 2020-09-12T13:58:51.049 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T13:58:18.155+0000 7fedee69c700 20 _merge_object_divergent_entries: merging hoid 1:c156033e:::existing_100:head entries: 29'101 (29'100) modify 1:c156033e:::existing_100:head by client.4524.0:1 2020-09-12T13:57:11.031720+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0 2020-09-12T13:58:51.049 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T13:58:18.155+0000 7fedee69c700 20 _merge_object_divergent_entries: keeping 29'101 (29'100) modify 1:c156033e:::existing_100:head by client.4524.0:1 2020-09-12T13:57:11.031720+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0 2020-09-12T13:58:51.049 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T13:58:18.155+0000 7fedee69c700 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2020-09-12T13:58:51.049 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T13:58:18.155+0000 7fedee69c700 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head prior_version: 29'100 first_divergent_update: 29'101 last_divergent_update: 29'101 2020-09-12T13:58:51.050 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T13:58:18.155+0000 7fedee69c700 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head has no more recent entries in log 2020-09-12T13:58:51.050 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T13:58:18.155+0000 7fedee69c700 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head must be rolled back or recovered, attempting to rollback 2020-09-12T13:58:51.050 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T13:58:18.155+0000 7fedee69c700 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head olog_can_rollback_to: 29'101 2020-09-12T13:58:51.050 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T13:58:18.155+0000 7fedee69c700 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot rollback 29'101 (29'100) modify 1:c156033e:::existing_100:head by client.4524.0:1 2020-09-12T13:57:11.031720+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0 2020-09-12T13:58:51.050 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T13:58:18.155+0000 7fedee69c700 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot roll back, removing and adding to missing 2020-09-12T13:58:51.051 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:216: TEST_divergent: find td/divergent-priors -name '*osd*log' 2020-09-12T13:58:51.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:216: TEST_divergent: grep -q '_merge_object_divergent_entries.*cannot roll back, removing and adding to missing' td/divergent-priors/osd.2.log td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log 2020-09-12T13:58:51.054 INFO:tasks.workunit.client.0.smithi027.stdout:success 2020-09-12T13:58:51.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:221: TEST_divergent: echo success 2020-09-12T13:58:51.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:223: TEST_divergent: delete_pool test 2020-09-12T13:58:51.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: delete_pool: local poolname=test 2020-09-12T13:58:51.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2020-09-12T13:58:51.597 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test' does not exist 2020-09-12T13:58:51.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:224: TEST_divergent: kill_daemons td/divergent-priors 2020-09-12T13:58:51.607 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T13:58:51.608 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T13:58:51.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T13:58:51.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T13:58:51.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T13:58:51.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T13:58:51.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2020-09-12T13:58:51.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2020-09-12T13:58:51.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T13:58:51.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2020-09-12T13:58:51.717 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T13:58:51.717 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T13:58:51.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T13:58:51.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T13:58:51.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T13:58:51.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T13:58:51.721 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T13:58:51.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T13:58:51.722 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T13:58:51.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T13:58:51.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T13:58:51.723 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T13:58:51.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T13:58:51.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T13:58:51.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T13:58:51.725 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T13:58:51.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T13:58:51.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T13:58:51.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2020-09-12T13:58:51.737 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T13:58:51.737 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:58:51.737 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:58:51.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.20815 2020-09-12T13:58:51.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T13:58:51.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T13:58:51.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2020-09-12T13:58:51.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2020-09-12T13:58:51.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/divergent-priors 2020-09-12T13:58:51.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/divergent-priors 2020-09-12T13:58:51.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2020-09-12T13:58:51.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T13:58:51.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2020-09-12T13:58:51.739 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T13:58:51.740 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T13:58:51.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T13:58:51.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T13:58:51.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T13:58:51.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T13:58:51.741 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T13:58:51.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T13:58:51.741 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T13:58:51.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T13:58:51.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T13:58:51.742 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T13:58:51.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T13:58:51.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T13:58:51.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T13:58:51.745 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T13:58:51.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T13:58:51.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T13:58:51.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2020-09-12T13:58:51.748 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T13:58:51.748 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:58:51.748 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:58:51.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.20815 2020-09-12T13:58:51.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T13:58:51.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T13:58:51.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/divergent-priors 2020-09-12T13:58:51.749 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2020-09-12T13:58:51.750 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:58:51.750 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:58:51.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.20815 2020-09-12T13:58:51.751 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2020-09-12T13:58:51.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2020-09-12T13:58:51.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2020-09-12T13:58:51.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2020-09-12T13:58:51.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent_2 td/divergent-priors 2020-09-12T13:58:51.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:421: TEST_divergent_2: local dir=td/divergent-priors 2020-09-12T13:58:51.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:424: TEST_divergent_2: local dummyfile=/etc/fstab 2020-09-12T13:58:51.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:425: TEST_divergent_2: local dummyfile2=/etc/resolv.conf 2020-09-12T13:58:51.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:427: TEST_divergent_2: local num_osds=3 2020-09-12T13:58:51.753 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:428: TEST_divergent_2: expr 3 - 1 2020-09-12T13:58:51.754 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:428: TEST_divergent_2: seq 0 2 2020-09-12T13:58:51.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:428: TEST_divergent_2: local 'osds=0 2020-09-12T13:58:51.754 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T13:58:51.754 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T13:58:51.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:429: TEST_divergent_2: run_mon td/divergent-priors a 2020-09-12T13:58:51.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/divergent-priors 2020-09-12T13:58:51.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2020-09-12T13:58:51.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2020-09-12T13:58:51.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2020-09-12T13:58:51.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/divergent-priors/a 2020-09-12T13:58:51.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2020-09-12T13:58:52.070 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2020-09-12T13:58:52.070 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T13:58:52.071 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T13:58:52.071 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T13:58:52.071 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:58:52.071 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:58:52.071 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:58:52.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 2020-09-12T13:58:52.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: cat 2020-09-12T13:58:52.255 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a fsid 2020-09-12T13:58:52.256 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T13:58:52.256 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T13:58:52.256 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=fsid 2020-09-12T13:58:52.257 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .fsid 2020-09-12T13:58:52.257 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T13:58:52.257 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T13:58:52.257 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T13:58:52.258 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T13:58:52.258 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:58:52.258 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:58:52.258 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-mon.a.asok 2020-09-12T13:58:52.258 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T13:58:52.258 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.20815/ceph-mon.a.asok config get fsid 2020-09-12T13:58:52.368 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a mon_host 2020-09-12T13:58:52.368 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T13:58:52.368 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T13:58:52.369 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=mon_host 2020-09-12T13:58:52.369 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .mon_host 2020-09-12T13:58:52.369 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T13:58:52.369 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T13:58:52.369 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T13:58:52.370 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T13:58:52.370 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:58:52.370 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:58:52.370 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-mon.a.asok 2020-09-12T13:58:52.371 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T13:58:52.371 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.20815/ceph-mon.a.asok config get mon_host 2020-09-12T13:58:52.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:430: TEST_divergent_2: run_mgr td/divergent-priors x 2020-09-12T13:58:52.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: local dir=td/divergent-priors 2020-09-12T13:58:52.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: shift 2020-09-12T13:58:52.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: local id=x 2020-09-12T13:58:52.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: shift 2020-09-12T13:58:52.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: local data=td/divergent-priors/x 2020-09-12T13:58:52.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:558: run_mgr: ceph config set mgr mgr/devicehealth/enable_monitoring off --force 2020-09-12T13:58:52.981 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: get_asok_path 2020-09-12T13:58:52.981 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T13:58:52.981 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T13:58:52.981 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T13:58:52.982 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:58:52.982 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:58:52.982 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:58:52.982 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T13:58:52.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T13:58:53.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:431: TEST_divergent_2: for i in $osds 2020-09-12T13:58:53.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:433: TEST_divergent_2: run_osd td/divergent-priors 0 2020-09-12T13:58:53.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/divergent-priors 2020-09-12T13:58:53.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T13:58:53.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=0 2020-09-12T13:58:53.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T13:58:53.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/divergent-priors/0 2020-09-12T13:58:53.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T13:58:53.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T13:58:53.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T13:58:53.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T13:58:53.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2020-09-12T13:58:53.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2020-09-12T13:58:53.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T13:58:53.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T13:58:53.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T13:58:53.012 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T13:58:53.012 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T13:58:53.012 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T13:58:53.013 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T13:58:53.013 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:58:53.013 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:58:53.013 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:58:53.014 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:58:53.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T13:58:53.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T13:58:53.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T13:58:53.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T13:58:53.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T13:58:53.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T13:58:53.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T13:58:53.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:58:53.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T13:58:53.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T13:58:53.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/divergent-priors/0 2020-09-12T13:58:53.017 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T13:58:53.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=6e938613-9665-406c-b148-ce5bfd24c111 2020-09-12T13:58:53.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd0 6e938613-9665-406c-b148-ce5bfd24c111' 2020-09-12T13:58:53.017 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T13:58:53.017 INFO:tasks.workunit.client.0.smithi027.stdout:add osd0 6e938613-9665-406c-b148-ce5bfd24c111 2020-09-12T13:58:53.032 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQAd1FxfSGbdARAAgOb8L4Ifj2vLU8u0PwSZBw== 2020-09-12T13:58:53.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQAd1FxfSGbdARAAgOb8L4Ifj2vLU8u0PwSZBw=="}' 2020-09-12T13:58:53.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 6e938613-9665-406c-b148-ce5bfd24c111 -i td/divergent-priors/0/new.json 2020-09-12T13:58:53.382 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T13:58:53.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/divergent-priors/0/new.json 2020-09-12T13:58:53.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 0 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQAd1FxfSGbdARAAgOb8L4Ifj2vLU8u0PwSZBw== --osd-uuid 6e938613-9665-406c-b148-ce5bfd24c111 2020-09-12T13:58:53.424 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:58:53.422+0000 7fb1c0c15ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:58:53.426 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:58:53.424+0000 7fb1c0c15ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:58:53.427 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:58:53.426+0000 7fb1c0c15ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:58:53.428 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:58:53.426+0000 7fb1c0c15ec0 -1 bluestore(td/divergent-priors/0/block) _read_bdev_label failed to open td/divergent-priors/0/block: (2) No such file or directory 2020-09-12T13:58:53.428 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:58:53.426+0000 7fb1c0c15ec0 -1 bluestore(td/divergent-priors/0/block) _read_bdev_label failed to open td/divergent-priors/0/block: (2) No such file or directory 2020-09-12T13:58:53.428 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:58:53.426+0000 7fb1c0c15ec0 -1 bluestore(td/divergent-priors/0/block) _read_bdev_label failed to open td/divergent-priors/0/block: (2) No such file or directory 2020-09-12T13:58:53.462 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:58:53.460+0000 7fb1c0c15ec0 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2020-09-12T13:58:58.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/divergent-priors/0/keyring 2020-09-12T13:58:58.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T13:58:58.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd0 key to auth repository 2020-09-12T13:58:58.009 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T13:58:58.009 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd0 key to auth repository 2020-09-12T13:58:58.563 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.0 2020-09-12T13:58:58.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.0 2020-09-12T13:58:58.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T13:58:58.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T13:58:58.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T13:58:58.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 0 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:58:58.593 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:58:58.591+0000 7f7a37db9ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:58:58.627 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:58:58.625+0000 7f7a37db9ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:58:58.628 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:58:58.627+0000 7f7a37db9ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:58:58.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 0 2020-09-12T13:58:58.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T13:58:58.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T13:58:58.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T13:58:58.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T13:58:58.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:58:58.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T13:58:58.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:58:58.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T13:58:58.977 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T13:58:59.144 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:58:59.142+0000 7f7a37db9ec0 -1 Falling back to public interface 2020-09-12T13:58:59.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T13:58:59.953 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:58:59.951+0000 7f7a37db9ec0 -1 osd.0 0 log_to_monitors {default=true} 2020-09-12T13:59:00.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T13:59:00.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:59:00.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T13:59:00.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:59:00.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T13:59:00.386 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T13:59:00.804 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/31596,v1:127.0.0.1:6803/31596] [v2:127.0.0.1:6804/31596,v1:127.0.0.1:6805/31596] exists,up 6e938613-9665-406c-b148-ce5bfd24c111 2020-09-12T13:59:00.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T13:59:00.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T13:59:00.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T13:59:00.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:431: TEST_divergent_2: for i in $osds 2020-09-12T13:59:00.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:433: TEST_divergent_2: run_osd td/divergent-priors 1 2020-09-12T13:59:00.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/divergent-priors 2020-09-12T13:59:00.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T13:59:00.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=1 2020-09-12T13:59:00.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T13:59:00.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/divergent-priors/1 2020-09-12T13:59:00.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T13:59:00.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T13:59:00.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T13:59:00.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T13:59:00.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2020-09-12T13:59:00.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2020-09-12T13:59:00.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T13:59:00.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T13:59:00.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T13:59:00.808 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T13:59:00.808 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T13:59:00.808 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T13:59:00.808 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T13:59:00.808 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:59:00.809 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:59:00.809 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:59:00.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:59:00.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T13:59:00.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T13:59:00.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T13:59:00.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T13:59:00.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T13:59:00.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T13:59:00.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T13:59:00.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:59:00.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T13:59:00.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T13:59:00.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/divergent-priors/1 2020-09-12T13:59:00.811 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T13:59:00.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=d92b7341-d129-4e21-85d9-c431d9f9298f 2020-09-12T13:59:00.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd1 d92b7341-d129-4e21-85d9-c431d9f9298f' 2020-09-12T13:59:00.812 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T13:59:00.812 INFO:tasks.workunit.client.0.smithi027.stdout:add osd1 d92b7341-d129-4e21-85d9-c431d9f9298f 2020-09-12T13:59:00.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQAk1FxftwspMRAALDU04iraYPPyCZbI3kpXGQ== 2020-09-12T13:59:00.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQAk1FxftwspMRAALDU04iraYPPyCZbI3kpXGQ=="}' 2020-09-12T13:59:00.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new d92b7341-d129-4e21-85d9-c431d9f9298f -i td/divergent-priors/1/new.json 2020-09-12T13:59:01.370 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T13:59:01.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/divergent-priors/1/new.json 2020-09-12T13:59:01.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 1 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQAk1FxftwspMRAALDU04iraYPPyCZbI3kpXGQ== --osd-uuid d92b7341-d129-4e21-85d9-c431d9f9298f 2020-09-12T13:59:01.412 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:01.411+0000 7f69cd32cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:59:01.414 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:01.412+0000 7f69cd32cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:59:01.415 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:01.414+0000 7f69cd32cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:59:01.416 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:01.414+0000 7f69cd32cec0 -1 bluestore(td/divergent-priors/1/block) _read_bdev_label failed to open td/divergent-priors/1/block: (2) No such file or directory 2020-09-12T13:59:01.416 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:01.414+0000 7f69cd32cec0 -1 bluestore(td/divergent-priors/1/block) _read_bdev_label failed to open td/divergent-priors/1/block: (2) No such file or directory 2020-09-12T13:59:01.416 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:01.414+0000 7f69cd32cec0 -1 bluestore(td/divergent-priors/1/block) _read_bdev_label failed to open td/divergent-priors/1/block: (2) No such file or directory 2020-09-12T13:59:01.493 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:01.492+0000 7f69cd32cec0 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2020-09-12T13:59:06.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/divergent-priors/1/keyring 2020-09-12T13:59:06.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T13:59:06.008 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd1 key to auth repository 2020-09-12T13:59:06.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd1 key to auth repository 2020-09-12T13:59:06.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T13:59:06.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.1 2020-09-12T13:59:06.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T13:59:06.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 1 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:59:06.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T13:59:06.566 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T13:59:06.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T13:59:06.594 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:06.593+0000 7ff41d90cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:59:06.598 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:06.597+0000 7ff41d90cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:59:06.600 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:06.599+0000 7ff41d90cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:59:06.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 1 2020-09-12T13:59:06.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T13:59:06.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T13:59:06.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T13:59:06.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T13:59:06.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:59:06.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T13:59:06.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:59:06.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T13:59:06.978 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T13:59:07.120 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:07.119+0000 7ff41d90cec0 -1 Falling back to public interface 2020-09-12T13:59:07.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T13:59:07.891 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:07.891+0000 7ff41d90cec0 -1 osd.1 0 log_to_monitors {default=true} 2020-09-12T13:59:08.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T13:59:08.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:59:08.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T13:59:08.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:59:08.384 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T13:59:08.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T13:59:08.823 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/32590,v1:127.0.0.1:6811/32590] [v2:127.0.0.1:6812/32590,v1:127.0.0.1:6813/32590] exists,up d92b7341-d129-4e21-85d9-c431d9f9298f 2020-09-12T13:59:08.823 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T13:59:08.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T13:59:08.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T13:59:08.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:431: TEST_divergent_2: for i in $osds 2020-09-12T13:59:08.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:433: TEST_divergent_2: run_osd td/divergent-priors 2 2020-09-12T13:59:08.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/divergent-priors 2020-09-12T13:59:08.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T13:59:08.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=2 2020-09-12T13:59:08.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T13:59:08.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/divergent-priors/2 2020-09-12T13:59:08.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T13:59:08.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T13:59:08.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T13:59:08.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T13:59:08.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2020-09-12T13:59:08.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2020-09-12T13:59:08.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T13:59:08.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T13:59:08.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T13:59:08.826 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T13:59:08.826 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T13:59:08.826 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T13:59:08.827 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T13:59:08.827 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:59:08.827 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:59:08.827 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:59:08.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T13:59:08.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T13:59:08.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T13:59:08.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T13:59:08.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T13:59:08.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T13:59:08.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T13:59:08.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T13:59:08.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:59:08.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T13:59:08.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T13:59:08.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/divergent-priors/2 2020-09-12T13:59:08.829 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T13:59:08.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=1b25b33b-61e6-4d39-ab58-fee24e94b77f 2020-09-12T13:59:08.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd2 1b25b33b-61e6-4d39-ab58-fee24e94b77f' 2020-09-12T13:59:08.831 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T13:59:08.831 INFO:tasks.workunit.client.0.smithi027.stdout:add osd2 1b25b33b-61e6-4d39-ab58-fee24e94b77f 2020-09-12T13:59:08.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQAs1Fxfi+U4MhAAjMpeIJnelPA/Cw4ov15g/A== 2020-09-12T13:59:08.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQAs1Fxfi+U4MhAAjMpeIJnelPA/Cw4ov15g/A=="}' 2020-09-12T13:59:08.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 1b25b33b-61e6-4d39-ab58-fee24e94b77f -i td/divergent-priors/2/new.json 2020-09-12T13:59:09.341 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T13:59:09.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/divergent-priors/2/new.json 2020-09-12T13:59:09.353 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 2 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQAs1Fxfi+U4MhAAjMpeIJnelPA/Cw4ov15g/A== --osd-uuid 1b25b33b-61e6-4d39-ab58-fee24e94b77f 2020-09-12T13:59:09.384 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:09.383+0000 7fd1877f4ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:59:09.385 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:09.385+0000 7fd1877f4ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:59:09.386 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:09.386+0000 7fd1877f4ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:59:09.387 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:09.386+0000 7fd1877f4ec0 -1 bluestore(td/divergent-priors/2/block) _read_bdev_label failed to open td/divergent-priors/2/block: (2) No such file or directory 2020-09-12T13:59:09.387 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:09.386+0000 7fd1877f4ec0 -1 bluestore(td/divergent-priors/2/block) _read_bdev_label failed to open td/divergent-priors/2/block: (2) No such file or directory 2020-09-12T13:59:09.387 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:09.386+0000 7fd1877f4ec0 -1 bluestore(td/divergent-priors/2/block) _read_bdev_label failed to open td/divergent-priors/2/block: (2) No such file or directory 2020-09-12T13:59:09.457 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:09.456+0000 7fd1877f4ec0 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2020-09-12T13:59:13.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/divergent-priors/2/keyring 2020-09-12T13:59:13.671 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T13:59:13.672 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd2 key to auth repository 2020-09-12T13:59:13.672 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd2 key to auth repository 2020-09-12T13:59:13.672 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T13:59:14.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.2 2020-09-12T13:59:14.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 2 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T13:59:14.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T13:59:14.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T13:59:14.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T13:59:14.228 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.2 2020-09-12T13:59:14.256 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:14.255+0000 7f648b749ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:59:14.280 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:14.279+0000 7f648b749ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:59:14.281 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:14.281+0000 7f648b749ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T13:59:14.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 2 2020-09-12T13:59:14.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T13:59:14.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T13:59:14.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T13:59:14.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T13:59:14.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:59:14.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T13:59:14.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:59:14.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T13:59:14.639 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T13:59:14.797 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:14.796+0000 7f648b749ec0 -1 Falling back to public interface 2020-09-12T13:59:15.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T13:59:15.581 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T13:59:15.580+0000 7f648b749ec0 -1 osd.2 0 log_to_monitors {default=true} 2020-09-12T13:59:16.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T13:59:16.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T13:59:16.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T13:59:16.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T13:59:16.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T13:59:16.049 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T13:59:16.467 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/33584,v1:127.0.0.1:6819/33584] [v2:127.0.0.1:6820/33584,v1:127.0.0.1:6821/33584] exists,up 1b25b33b-61e6-4d39-ab58-fee24e94b77f 2020-09-12T13:59:16.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T13:59:16.468 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T13:59:16.468 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T13:59:16.468 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:436: TEST_divergent_2: ceph osd set noout 2020-09-12T13:59:17.217 INFO:tasks.workunit.client.0.smithi027.stderr:noout is set 2020-09-12T13:59:17.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:437: TEST_divergent_2: ceph osd set noin 2020-09-12T13:59:18.019 INFO:tasks.workunit.client.0.smithi027.stderr:noin is set 2020-09-12T13:59:18.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:438: TEST_divergent_2: ceph osd set nodown 2020-09-12T13:59:18.757 INFO:tasks.workunit.client.0.smithi027.stderr:nodown is set 2020-09-12T13:59:18.766 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:439: TEST_divergent_2: create_pool test 1 1 2020-09-12T13:59:18.767 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test 1 1 2020-09-12T13:59:19.334 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test' already exists 2020-09-12T13:59:19.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T13:59:20.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:440: TEST_divergent_2: ceph osd pool set test size 3 2020-09-12T13:59:21.099 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 1 size to 3 2020-09-12T13:59:21.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:441: TEST_divergent_2: ceph osd pool set test min_size 2 2020-09-12T13:59:21.985 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 1 min_size to 2 2020-09-12T13:59:21.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:443: TEST_divergent_2: flush_pg_stats 2020-09-12T13:59:21.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T13:59:21.995 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T13:59:22.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T13:59:22.428 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T13:59:22.428 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T13:59:22.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T13:59:22.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T13:59:22.428 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T13:59:22.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836487 2020-09-12T13:59:22.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836487 2020-09-12T13:59:22.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836487' 2020-09-12T13:59:22.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T13:59:22.576 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T13:59:22.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672965 2020-09-12T13:59:22.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672965 2020-09-12T13:59:22.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836487 1-42949672965' 2020-09-12T13:59:22.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T13:59:22.724 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T13:59:22.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509444 2020-09-12T13:59:22.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509444 2020-09-12T13:59:22.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836487 1-42949672965 2-64424509444' 2020-09-12T13:59:22.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T13:59:22.871 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836487 2020-09-12T13:59:22.871 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T13:59:22.871 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T13:59:22.871 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836487 2020-09-12T13:59:22.872 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T13:59:22.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836487 2020-09-12T13:59:22.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2020-09-12T13:59:22.872 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T13:59:22.873 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836487 2020-09-12T13:59:23.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836487 -lt 21474836487 2020-09-12T13:59:23.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T13:59:23.278 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672965 2020-09-12T13:59:23.278 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T13:59:23.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T13:59:23.279 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672965 2020-09-12T13:59:23.279 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T13:59:23.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672965 2020-09-12T13:59:23.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672965' 2020-09-12T13:59:23.281 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672965 2020-09-12T13:59:23.281 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T13:59:23.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672965 -lt 42949672965 2020-09-12T13:59:23.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T13:59:23.710 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509444 2020-09-12T13:59:23.710 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T13:59:23.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T13:59:23.718 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509444 2020-09-12T13:59:23.719 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T13:59:23.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509444 2020-09-12T13:59:23.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509444' 2020-09-12T13:59:23.719 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T13:59:23.720 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509444 2020-09-12T13:59:24.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509442 -lt 64424509444 2020-09-12T13:59:24.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T13:59:25.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T13:59:25.122 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T13:59:25.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509444 -lt 64424509444 2020-09-12T13:59:25.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:444: TEST_divergent_2: wait_for_clean 2020-09-12T13:59:25.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T13:59:25.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T13:59:25.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T13:59:25.547 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T13:59:25.547 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 90 .1 2020-09-12T13:59:25.547 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T13:59:25.548 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T13:59:25.548 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T13:59:25.548 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T13:59:25.548 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T13:59:25.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T13:59:25.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T13:59:25.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T13:59:25.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T13:59:25.642 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T13:59:26.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T13:59:26.055 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T13:59:26.056 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T13:59:26.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T13:59:26.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T13:59:26.056 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T13:59:26.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836489 2020-09-12T13:59:26.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836489 2020-09-12T13:59:26.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836489' 2020-09-12T13:59:26.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T13:59:26.207 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T13:59:26.354 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672968 2020-09-12T13:59:26.354 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672968 2020-09-12T13:59:26.354 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836489 1-42949672968' 2020-09-12T13:59:26.355 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T13:59:26.355 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T13:59:26.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509446 2020-09-12T13:59:26.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509446 2020-09-12T13:59:26.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836489 1-42949672968 2-64424509446' 2020-09-12T13:59:26.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T13:59:26.502 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836489 2020-09-12T13:59:26.502 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T13:59:26.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T13:59:26.503 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836489 2020-09-12T13:59:26.503 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T13:59:26.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836489 2020-09-12T13:59:26.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2020-09-12T13:59:26.504 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T13:59:26.505 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836489 2020-09-12T13:59:26.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836490 -lt 21474836489 2020-09-12T13:59:26.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T13:59:26.931 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672968 2020-09-12T13:59:26.932 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T13:59:26.932 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T13:59:26.932 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672968 2020-09-12T13:59:26.933 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T13:59:26.933 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672968 2020-09-12T13:59:26.934 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672968' 2020-09-12T13:59:26.934 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672968 2020-09-12T13:59:26.934 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T13:59:27.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672968 -lt 42949672968 2020-09-12T13:59:27.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T13:59:27.349 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509446 2020-09-12T13:59:27.349 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T13:59:27.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T13:59:27.350 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509446 2020-09-12T13:59:27.351 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T13:59:27.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509446 2020-09-12T13:59:27.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509446' 2020-09-12T13:59:27.352 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T13:59:27.353 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509446 2020-09-12T13:59:27.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509446 -lt 64424509446 2020-09-12T13:59:27.782 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T13:59:27.782 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T13:59:27.782 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T13:59:28.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 1 == 0 2020-09-12T13:59:28.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T13:59:28.321 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T13:59:28.321 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T13:59:28.321 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T13:59:28.322 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T13:59:28.322 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T13:59:28.322 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T13:59:28.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T13:59:28.716 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T13:59:28.717 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T13:59:28.717 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T13:59:29.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 1 2020-09-12T13:59:29.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T13:59:29.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T13:59:29.258 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:447: TEST_divergent_2: ceph pg dump pgs --format=json 2020-09-12T13:59:29.258 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:447: TEST_divergent_2: jq '.pg_stats[0].up_primary' 2020-09-12T13:59:29.645 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T13:59:29.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:447: TEST_divergent_2: local divergent=1 2020-09-12T13:59:29.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:448: TEST_divergent_2: echo 'primary and soon to be divergent is 1' 2020-09-12T13:59:29.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:449: TEST_divergent_2: ceph pg dump pgs 2020-09-12T13:59:29.655 INFO:tasks.workunit.client.0.smithi027.stdout:primary and soon to be divergent is 1 2020-09-12T13:59:30.035 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T13:59:30.035 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 0 0 0 0 0 0 0 0 0 0 active+clean 2020-09-12T13:59:20.268313+0000 0'0 28:12 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T13:59:19.138081+0000 0'0 2020-09-12T13:59:19.138081+0000 0 2020-09-12T13:59:30.036 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T13:59:30.036 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T13:59:30.037 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T13:59:30.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:450: TEST_divergent_2: local non_divergent= 2020-09-12T13:59:30.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:451: TEST_divergent_2: for i in $osds 2020-09-12T13:59:30.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:453: TEST_divergent_2: '[' 0 = 1 ']' 2020-09-12T13:59:30.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:456: TEST_divergent_2: non_divergent=' 0' 2020-09-12T13:59:30.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:451: TEST_divergent_2: for i in $osds 2020-09-12T13:59:30.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:453: TEST_divergent_2: '[' 1 = 1 ']' 2020-09-12T13:59:30.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:454: TEST_divergent_2: continue 2020-09-12T13:59:30.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:451: TEST_divergent_2: for i in $osds 2020-09-12T13:59:30.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:453: TEST_divergent_2: '[' 2 = 1 ']' 2020-09-12T13:59:30.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:456: TEST_divergent_2: non_divergent=' 0 2' 2020-09-12T13:59:30.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:459: TEST_divergent_2: echo 'writing initial objects' 2020-09-12T13:59:30.048 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: seq 1 100 2020-09-12T13:59:30.049 INFO:tasks.workunit.client.0.smithi027.stdout:writing initial objects 2020-09-12T13:59:30.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:30.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_1 /etc/fstab 2020-09-12T13:59:30.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:30.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_2 /etc/fstab 2020-09-12T13:59:30.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:30.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_3 /etc/fstab 2020-09-12T13:59:30.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:30.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_4 /etc/fstab 2020-09-12T13:59:30.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:30.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_5 /etc/fstab 2020-09-12T13:59:30.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:30.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_6 /etc/fstab 2020-09-12T13:59:30.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:30.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_7 /etc/fstab 2020-09-12T13:59:31.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:31.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_8 /etc/fstab 2020-09-12T13:59:31.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:31.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_9 /etc/fstab 2020-09-12T13:59:31.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:31.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_10 /etc/fstab 2020-09-12T13:59:31.510 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:31.510 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_11 /etc/fstab 2020-09-12T13:59:31.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:31.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_12 /etc/fstab 2020-09-12T13:59:31.785 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:31.785 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_13 /etc/fstab 2020-09-12T13:59:31.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:31.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_14 /etc/fstab 2020-09-12T13:59:32.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:32.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_15 /etc/fstab 2020-09-12T13:59:32.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:32.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_16 /etc/fstab 2020-09-12T13:59:32.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:32.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_17 /etc/fstab 2020-09-12T13:59:32.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:32.433 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_18 /etc/fstab 2020-09-12T13:59:32.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:32.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_19 /etc/fstab 2020-09-12T13:59:32.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:32.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_20 /etc/fstab 2020-09-12T13:59:32.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:32.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_21 /etc/fstab 2020-09-12T13:59:32.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:32.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_22 /etc/fstab 2020-09-12T13:59:33.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:33.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_23 /etc/fstab 2020-09-12T13:59:33.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:33.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_24 /etc/fstab 2020-09-12T13:59:33.423 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:33.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_25 /etc/fstab 2020-09-12T13:59:33.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:33.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_26 /etc/fstab 2020-09-12T13:59:33.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:33.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_27 /etc/fstab 2020-09-12T13:59:33.859 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:33.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_28 /etc/fstab 2020-09-12T13:59:33.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:33.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_29 /etc/fstab 2020-09-12T13:59:34.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:34.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_30 /etc/fstab 2020-09-12T13:59:34.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:34.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_31 /etc/fstab 2020-09-12T13:59:34.438 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:34.438 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_32 /etc/fstab 2020-09-12T13:59:34.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:34.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_33 /etc/fstab 2020-09-12T13:59:34.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:34.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_34 /etc/fstab 2020-09-12T13:59:34.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:34.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_35 /etc/fstab 2020-09-12T13:59:35.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:35.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_36 /etc/fstab 2020-09-12T13:59:35.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:35.151 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_37 /etc/fstab 2020-09-12T13:59:35.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:35.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_38 /etc/fstab 2020-09-12T13:59:35.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:35.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_39 /etc/fstab 2020-09-12T13:59:35.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:35.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_40 /etc/fstab 2020-09-12T13:59:35.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:35.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_41 /etc/fstab 2020-09-12T13:59:35.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:35.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_42 /etc/fstab 2020-09-12T13:59:36.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:36.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_43 /etc/fstab 2020-09-12T13:59:36.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:36.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_44 /etc/fstab 2020-09-12T13:59:36.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:36.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_45 /etc/fstab 2020-09-12T13:59:36.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:36.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_46 /etc/fstab 2020-09-12T13:59:36.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:36.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_47 /etc/fstab 2020-09-12T13:59:36.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:36.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_48 /etc/fstab 2020-09-12T13:59:36.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:36.894 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_49 /etc/fstab 2020-09-12T13:59:37.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:37.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_50 /etc/fstab 2020-09-12T13:59:37.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:37.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_51 /etc/fstab 2020-09-12T13:59:37.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:37.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_52 /etc/fstab 2020-09-12T13:59:37.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:37.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_53 /etc/fstab 2020-09-12T13:59:37.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:37.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_54 /etc/fstab 2020-09-12T13:59:37.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:37.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_55 /etc/fstab 2020-09-12T13:59:37.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:37.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_56 /etc/fstab 2020-09-12T13:59:38.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:38.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_57 /etc/fstab 2020-09-12T13:59:38.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:38.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_58 /etc/fstab 2020-09-12T13:59:38.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:38.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_59 /etc/fstab 2020-09-12T13:59:38.444 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:38.444 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_60 /etc/fstab 2020-09-12T13:59:38.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:38.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_61 /etc/fstab 2020-09-12T13:59:38.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:38.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_62 /etc/fstab 2020-09-12T13:59:38.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:38.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_63 /etc/fstab 2020-09-12T13:59:39.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:39.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_64 /etc/fstab 2020-09-12T13:59:39.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:39.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_65 /etc/fstab 2020-09-12T13:59:39.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:39.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_66 /etc/fstab 2020-09-12T13:59:39.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:39.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_67 /etc/fstab 2020-09-12T13:59:39.550 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:39.550 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_68 /etc/fstab 2020-09-12T13:59:39.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:39.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_69 /etc/fstab 2020-09-12T13:59:39.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:39.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_70 /etc/fstab 2020-09-12T13:59:39.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:39.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_71 /etc/fstab 2020-09-12T13:59:40.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:40.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_72 /etc/fstab 2020-09-12T13:59:40.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:40.246 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_73 /etc/fstab 2020-09-12T13:59:40.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:40.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_74 /etc/fstab 2020-09-12T13:59:40.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:40.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_75 /etc/fstab 2020-09-12T13:59:40.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:40.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_76 /etc/fstab 2020-09-12T13:59:40.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:40.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_77 /etc/fstab 2020-09-12T13:59:41.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:41.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_78 /etc/fstab 2020-09-12T13:59:41.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:41.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_79 /etc/fstab 2020-09-12T13:59:41.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:41.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_80 /etc/fstab 2020-09-12T13:59:41.444 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:41.445 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_81 /etc/fstab 2020-09-12T13:59:41.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:41.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_82 /etc/fstab 2020-09-12T13:59:41.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:41.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_83 /etc/fstab 2020-09-12T13:59:41.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:41.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_84 /etc/fstab 2020-09-12T13:59:41.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:41.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_85 /etc/fstab 2020-09-12T13:59:42.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:42.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_86 /etc/fstab 2020-09-12T13:59:42.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:42.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_87 /etc/fstab 2020-09-12T13:59:42.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:42.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_88 /etc/fstab 2020-09-12T13:59:42.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:42.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_89 /etc/fstab 2020-09-12T13:59:42.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:42.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_90 /etc/fstab 2020-09-12T13:59:42.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:42.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_91 /etc/fstab 2020-09-12T13:59:42.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:42.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_92 /etc/fstab 2020-09-12T13:59:43.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:43.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_93 /etc/fstab 2020-09-12T13:59:43.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:43.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_94 /etc/fstab 2020-09-12T13:59:43.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:43.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_95 /etc/fstab 2020-09-12T13:59:43.548 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:43.548 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_96 /etc/fstab 2020-09-12T13:59:43.665 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:43.665 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_97 /etc/fstab 2020-09-12T13:59:43.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:43.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_98 /etc/fstab 2020-09-12T13:59:43.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:43.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_99 /etc/fstab 2020-09-12T13:59:44.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2020-09-12T13:59:44.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_100 /etc/fstab 2020-09-12T13:59:44.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:466: TEST_divergent_2: WAIT_FOR_CLEAN_TIMEOUT=20 2020-09-12T13:59:44.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:466: TEST_divergent_2: wait_for_clean 2020-09-12T13:59:44.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T13:59:44.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T13:59:44.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T13:59:44.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T13:59:44.244 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 20 .1 2020-09-12T13:59:44.245 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T13:59:44.245 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T13:59:44.245 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T13:59:44.246 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T13:59:44.246 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T13:59:44.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T13:59:44.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T13:59:44.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T13:59:44.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T13:59:44.307 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T13:59:44.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T13:59:44.715 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T13:59:44.715 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T13:59:44.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T13:59:44.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T13:59:44.716 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T13:59:44.862 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836495 2020-09-12T13:59:44.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836495 2020-09-12T13:59:44.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836495' 2020-09-12T13:59:44.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T13:59:44.863 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T13:59:45.009 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672974 2020-09-12T13:59:45.009 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672974 2020-09-12T13:59:45.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836495 1-42949672974' 2020-09-12T13:59:45.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T13:59:45.010 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T13:59:45.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509452 2020-09-12T13:59:45.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509452 2020-09-12T13:59:45.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836495 1-42949672974 2-64424509452' 2020-09-12T13:59:45.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T13:59:45.157 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836495 2020-09-12T13:59:45.157 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T13:59:45.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T13:59:45.158 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836495 2020-09-12T13:59:45.158 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T13:59:45.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836495 2020-09-12T13:59:45.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836495' 2020-09-12T13:59:45.159 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T13:59:45.160 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836495 2020-09-12T13:59:45.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836493 -lt 21474836495 2020-09-12T13:59:45.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T13:59:46.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T13:59:46.568 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T13:59:46.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836496 -lt 21474836495 2020-09-12T13:59:46.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T13:59:46.986 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672974 2020-09-12T13:59:46.986 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T13:59:46.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T13:59:46.987 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672974 2020-09-12T13:59:46.988 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T13:59:46.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672974 2020-09-12T13:59:46.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672974' 2020-09-12T13:59:46.989 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672974 2020-09-12T13:59:46.989 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T13:59:47.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672974 -lt 42949672974 2020-09-12T13:59:47.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T13:59:47.394 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509452 2020-09-12T13:59:47.395 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T13:59:47.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T13:59:47.395 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509452 2020-09-12T13:59:47.396 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T13:59:47.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509452 2020-09-12T13:59:47.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509452' 2020-09-12T13:59:47.397 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509452 2020-09-12T13:59:47.397 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T13:59:47.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509452 -lt 64424509452 2020-09-12T13:59:47.806 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T13:59:47.806 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T13:59:47.807 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T13:59:48.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 1 == 0 2020-09-12T13:59:48.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T13:59:48.346 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T13:59:48.346 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T13:59:48.346 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T13:59:48.346 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T13:59:48.346 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T13:59:48.346 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T13:59:48.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T13:59:48.740 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T13:59:48.740 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T13:59:48.741 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T13:59:49.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 1 2020-09-12T13:59:49.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T13:59:49.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T13:59:49.297 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:468: TEST_divergent_2: get_pg test existing_1 2020-09-12T13:59:49.297 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1031: get_pg: local poolname=test 2020-09-12T13:59:49.297 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1032: get_pg: local objectname=existing_1 2020-09-12T13:59:49.297 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1034: get_pg: ceph --format json osd map test existing_1 2020-09-12T13:59:49.298 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1034: get_pg: jq -r .pgid 2020-09-12T13:59:49.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:468: TEST_divergent_2: local pgid=1.0 2020-09-12T13:59:49.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:471: TEST_divergent_2: echo 'blackholing osds 0 2' 2020-09-12T13:59:49.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:472: TEST_divergent_2: ceph pg dump pgs 2020-09-12T13:59:49.734 INFO:tasks.workunit.client.0.smithi027.stdout:blackholing osds 0 2 2020-09-12T13:59:50.148 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T13:59:50.148 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 0 0 0 0 48300 0 0 16 16 active+clean 2020-09-12T13:59:20.268313+0000 29'100 29:112 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T13:59:19.138081+0000 0'0 2020-09-12T13:59:19.138081+0000 0 2020-09-12T13:59:50.149 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T13:59:50.149 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T13:59:50.150 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T13:59:50.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:473: TEST_divergent_2: for i in $non_divergent 2020-09-12T13:59:50.159 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: get_asok_path osd.0 2020-09-12T13:59:50.160 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2020-09-12T13:59:50.160 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2020-09-12T13:59:50.160 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T13:59:50.161 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:59:50.161 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:59:50.161 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.0.asok 2020-09-12T13:59:50.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: CEPH_ARGS= 2020-09-12T13:59:50.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.0.asok config set objectstore_blackhole 1 2020-09-12T13:59:50.262 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T13:59:50.262 INFO:tasks.workunit.client.0.smithi027.stdout: "success": "objectstore_blackhole = 'true' (not observed, change may require restart) " 2020-09-12T13:59:50.263 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T13:59:50.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:473: TEST_divergent_2: for i in $non_divergent 2020-09-12T13:59:50.271 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: get_asok_path osd.2 2020-09-12T13:59:50.271 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2020-09-12T13:59:50.271 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2020-09-12T13:59:50.271 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T13:59:50.272 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T13:59:50.272 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T13:59:50.272 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.2.asok 2020-09-12T13:59:50.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: CEPH_ARGS= 2020-09-12T13:59:50.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.2.asok config set objectstore_blackhole 1 2020-09-12T13:59:50.373 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T13:59:50.374 INFO:tasks.workunit.client.0.smithi027.stdout: "success": "objectstore_blackhole = 'true' (not observed, change may require restart) " 2020-09-12T13:59:50.374 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T13:59:50.381 INFO:tasks.workunit.client.0.smithi027.stdout:create new divergent objects 2020-09-12T13:59:50.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:479: TEST_divergent_2: echo 'create new divergent objects' 2020-09-12T13:59:50.382 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:480: TEST_divergent_2: seq 1 2 2020-09-12T13:59:50.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:480: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2020-09-12T13:59:50.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:480: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2020-09-12T13:59:50.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:482: TEST_divergent_2: rados -p test create newobject_1 2020-09-12T13:59:50.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:485: TEST_divergent_2: echo 'writing divergent objects' 2020-09-12T13:59:50.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:482: TEST_divergent_2: rados -p test create newobject_2 2020-09-12T13:59:50.384 INFO:tasks.workunit.client.0.smithi027.stdout:writing divergent objects 2020-09-12T13:59:50.384 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:486: TEST_divergent_2: seq 1 2 2020-09-12T13:59:50.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:486: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_WRITE) 2020-09-12T13:59:50.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:486: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_WRITE) 2020-09-12T13:59:50.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:488: TEST_divergent_2: rados -p test put existing_1 /etc/resolv.conf 2020-09-12T13:59:50.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:491: TEST_divergent_2: echo 'remove divergent objects' 2020-09-12T13:59:50.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:488: TEST_divergent_2: rados -p test put existing_2 /etc/resolv.conf 2020-09-12T13:59:50.385 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:492: TEST_divergent_2: seq 1 2 2020-09-12T13:59:50.385 INFO:tasks.workunit.client.0.smithi027.stdout:remove divergent objects 2020-09-12T13:59:50.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:492: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_REMOVE) 2020-09-12T13:59:50.386 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:494: TEST_divergent_2: expr 1 + 2 2020-09-12T13:59:50.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:494: TEST_divergent_2: rmi=3 2020-09-12T13:59:50.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:492: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_REMOVE) 2020-09-12T13:59:50.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:495: TEST_divergent_2: rados -p test rm existing_3 2020-09-12T13:59:50.388 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:494: TEST_divergent_2: expr 2 + 2 2020-09-12T13:59:50.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:494: TEST_divergent_2: rmi=4 2020-09-12T13:59:50.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:497: TEST_divergent_2: sleep 10 2020-09-12T13:59:50.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:495: TEST_divergent_2: rados -p test rm existing_4 2020-09-12T14:00:00.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:498: TEST_divergent_2: killall -9 rados 2020-09-12T14:00:00.393 INFO:tasks.workunit.client.0.smithi027.stderr:rados: no process found 2020-09-12T14:00:00.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:501: TEST_divergent_2: echo 'killing all the osds' 2020-09-12T14:00:00.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:502: TEST_divergent_2: ceph pg dump pgs 2020-09-12T14:00:00.393 INFO:tasks.workunit.client.0.smithi027.stdout:killing all the osds 2020-09-12T14:00:00.795 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:00:00.796 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 0 0 0 0 46628 0 0 16 16 active+clean 2020-09-12T13:59:20.268313+0000 29'106 29:118 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T13:59:19.138081+0000 0'0 2020-09-12T13:59:19.138081+0000 0 2020-09-12T14:00:00.796 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:00:00.796 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:00:00.797 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:00:00.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:503: TEST_divergent_2: kill_daemons td/divergent-priors KILL osd 2020-09-12T14:00:00.807 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:00:00.807 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:00:00.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:00:00.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:00:00.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:00:00.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:00:00.914 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:504: TEST_divergent_2: for i in $osds 2020-09-12T14:00:00.914 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:506: TEST_divergent_2: ceph osd down osd.0 2020-09-12T14:00:01.516 INFO:tasks.workunit.client.0.smithi027.stderr:osd.0 is already down. 2020-09-12T14:00:01.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:504: TEST_divergent_2: for i in $osds 2020-09-12T14:00:01.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:506: TEST_divergent_2: ceph osd down osd.1 2020-09-12T14:00:02.085 INFO:tasks.workunit.client.0.smithi027.stderr:osd.1 is already down. 2020-09-12T14:00:02.094 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:504: TEST_divergent_2: for i in $osds 2020-09-12T14:00:02.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:506: TEST_divergent_2: ceph osd down osd.2 2020-09-12T14:00:02.672 INFO:tasks.workunit.client.0.smithi027.stderr:osd.2 is already down. 2020-09-12T14:00:02.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:508: TEST_divergent_2: for i in $non_divergent 2020-09-12T14:00:02.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:510: TEST_divergent_2: ceph osd out osd.0 2020-09-12T14:00:03.260 INFO:tasks.workunit.client.0.smithi027.stderr:osd.0 is already out. 2020-09-12T14:00:03.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:508: TEST_divergent_2: for i in $non_divergent 2020-09-12T14:00:03.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:510: TEST_divergent_2: ceph osd out osd.2 2020-09-12T14:00:03.820 INFO:tasks.workunit.client.0.smithi027.stderr:osd.2 is already out. 2020-09-12T14:00:03.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:514: TEST_divergent_2: echo 'bringing up non_divergent 0 2' 2020-09-12T14:00:03.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:515: TEST_divergent_2: ceph pg dump pgs 2020-09-12T14:00:03.830 INFO:tasks.workunit.client.0.smithi027.stdout:bringing up non_divergent 0 2 2020-09-12T14:00:04.215 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:00:04.216 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 0 0 0 0 46628 0 0 16 16 stale+active+clean 2020-09-12T13:59:20.268313+0000 29'106 29:118 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T13:59:19.138081+0000 0'0 2020-09-12T13:59:19.138081+0000 0 2020-09-12T14:00:04.216 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:00:04.216 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:00:04.217 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:00:04.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:516: TEST_divergent_2: for i in $non_divergent 2020-09-12T14:00:04.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:518: TEST_divergent_2: activate_osd td/divergent-priors 0 2020-09-12T14:00:04.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/divergent-priors 2020-09-12T14:00:04.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T14:00:04.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=0 2020-09-12T14:00:04.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T14:00:04.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/divergent-priors/0 2020-09-12T14:00:04.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T14:00:04.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:00:04.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:00:04.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:00:04.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2020-09-12T14:00:04.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2020-09-12T14:00:04.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T14:00:04.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T14:00:04.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T14:00:04.230 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T14:00:04.230 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:00:04.230 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:00:04.231 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:00:04.231 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:00:04.231 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:00:04.231 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:00:04.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:00:04.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:00:04.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T14:00:04.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T14:00:04.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:00:04.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:00:04.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:00:04.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T14:00:04.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T14:00:04.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/divergent-priors/0 2020-09-12T14:00:04.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.0 2020-09-12T14:00:04.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 0 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:00:04.234 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/divergent-priors/0/whoami 2020-09-12T14:00:04.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 0 = 0 ']' 2020-09-12T14:00:04.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T14:00:04.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T14:00:04.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T14:00:04.235 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.0 2020-09-12T14:00:04.258 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:00:04.258+0000 7fc85640aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:00:04.260 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:00:04.259+0000 7fc85640aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:00:04.261 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:00:04.260+0000 7fc85640aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:00:04.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: wait_for_osd up 0 2020-09-12T14:00:04.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:00:04.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T14:00:04.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:00:04.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:00:04.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:00:04.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:00:04.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:00:04.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:00:04.642 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:00:04.777 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:00:04.775+0000 7fc85640aec0 -1 Falling back to public interface 2020-09-12T14:00:05.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:00:05.954 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:00:05.952+0000 7fc85640aec0 -1 osd.0 29 log_to_monitors {default=true} 2020-09-12T14:00:06.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:00:06.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:00:06.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:00:06.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:00:06.078 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:00:06.079 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:00:06.522 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up out weight 0 up_from 35 up_thru 0 down_at 30 last_clean_interval [5,29) [v2:127.0.0.1:6802/37498,v1:127.0.0.1:6803/37498] [v2:127.0.0.1:6804/37498,v1:127.0.0.1:6805/37498] exists,up 6e938613-9665-406c-b148-ce5bfd24c111 2020-09-12T14:00:06.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:00:06.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:00:06.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:00:06.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:516: TEST_divergent_2: for i in $non_divergent 2020-09-12T14:00:06.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:518: TEST_divergent_2: activate_osd td/divergent-priors 2 2020-09-12T14:00:06.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/divergent-priors 2020-09-12T14:00:06.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T14:00:06.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=2 2020-09-12T14:00:06.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T14:00:06.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/divergent-priors/2 2020-09-12T14:00:06.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T14:00:06.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:00:06.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:00:06.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:00:06.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2020-09-12T14:00:06.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2020-09-12T14:00:06.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T14:00:06.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T14:00:06.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T14:00:06.525 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T14:00:06.525 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:00:06.525 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:00:06.525 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:00:06.525 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:00:06.525 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:00:06.525 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:00:06.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:00:06.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:00:06.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T14:00:06.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T14:00:06.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:00:06.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:00:06.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:00:06.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T14:00:06.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T14:00:06.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/divergent-priors/2 2020-09-12T14:00:06.528 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.2 2020-09-12T14:00:06.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.2 2020-09-12T14:00:06.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 2 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:00:06.528 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/divergent-priors/2/whoami 2020-09-12T14:00:06.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 2 = 2 ']' 2020-09-12T14:00:06.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T14:00:06.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T14:00:06.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T14:00:06.551 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:00:06.549+0000 7f4e857d2ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:00:06.553 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:00:06.551+0000 7f4e857d2ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:00:06.554 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:00:06.552+0000 7f4e857d2ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:00:06.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: wait_for_osd up 2 2020-09-12T14:00:06.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:00:06.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T14:00:06.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:00:06.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:00:06.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:00:06.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:00:06.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:00:06.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:00:06.951 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:00:07.073 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:00:07.071+0000 7f4e857d2ec0 -1 Falling back to public interface 2020-09-12T14:00:07.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:00:08.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:00:08.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:00:08.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:00:08.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:00:08.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:00:08.360 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:00:08.380 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:00:08.378+0000 7f4e857d2ec0 -1 osd.2 29 log_to_monitors {default=true} 2020-09-12T14:00:08.791 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up out weight 0 up_from 36 up_thru 0 down_at 32 last_clean_interval [15,29) [v2:127.0.0.1:6810/37897,v1:127.0.0.1:6811/37897] [v2:127.0.0.1:6812/37897,v1:127.0.0.1:6813/37897] exists,up 1b25b33b-61e6-4d39-ab58-fee24e94b77f 2020-09-12T14:00:08.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:00:08.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:00:08.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:00:08.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:520: TEST_divergent_2: for i in $non_divergent 2020-09-12T14:00:08.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:522: TEST_divergent_2: ceph osd in osd.0 2020-09-12T14:00:09.359 INFO:tasks.workunit.client.0.smithi027.stderr:osd.0 is already in. 2020-09-12T14:00:09.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:520: TEST_divergent_2: for i in $non_divergent 2020-09-12T14:00:09.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:522: TEST_divergent_2: ceph osd in osd.2 2020-09-12T14:00:09.929 INFO:tasks.workunit.client.0.smithi027.stderr:osd.2 is already in. 2020-09-12T14:00:09.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:525: TEST_divergent_2: WAIT_FOR_CLEAN_TIMEOUT=20 2020-09-12T14:00:09.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:525: TEST_divergent_2: wait_for_clean 2020-09-12T14:00:09.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:00:09.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:00:09.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:00:09.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:00:09.940 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 20 .1 2020-09-12T14:00:09.940 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:00:09.941 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:00:09.941 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:00:09.941 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:00:09.941 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:00:09.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:00:09.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:00:09.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:00:09.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:00:09.999 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:00:10.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:00:10.484 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:00:10.484 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T14:00:10.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:00:10.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:00:10.485 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:00:10.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=150323855363 2020-09-12T14:00:10.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 150323855363 2020-09-12T14:00:10.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-150323855363' 2020-09-12T14:00:10.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:00:10.634 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:00:10.740 INFO:tasks.workunit.client.0.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-09-12T14:00:10.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq= 2020-09-12T14:00:10.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z '' 2020-09-12T14:00:10.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2075: flush_pg_stats: continue 2020-09-12T14:00:10.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:00:10.742 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:00:10.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=154618822659 2020-09-12T14:00:10.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 154618822659 2020-09-12T14:00:10.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-150323855363 2-154618822659' 2020-09-12T14:00:10.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:00:10.891 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-150323855363 2020-09-12T14:00:10.892 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:00:10.892 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:00:10.892 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-150323855363 2020-09-12T14:00:10.892 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:00:10.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=150323855363 2020-09-12T14:00:10.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 150323855363' 2020-09-12T14:00:10.893 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:00:10.894 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 150323855363 2020-09-12T14:00:11.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 150323855363 -lt 150323855363 2020-09-12T14:00:11.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:00:11.305 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-154618822659 2020-09-12T14:00:11.305 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:00:11.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:00:11.306 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-154618822659 2020-09-12T14:00:11.307 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:00:11.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=154618822659 2020-09-12T14:00:11.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 154618822659' 2020-09-12T14:00:11.308 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 154618822659 2020-09-12T14:00:11.308 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:00:11.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 1 -lt 154618822659 2020-09-12T14:00:11.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:00:12.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:00:12.738 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:00:13.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 154618822659 -lt 154618822659 2020-09-12T14:00:13.163 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:00:13.164 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:00:13.164 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:00:13.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 1 == 0 2020-09-12T14:00:13.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:00:13.714 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:00:13.714 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:00:13.715 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:00:13.715 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:00:13.715 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:00:13.715 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:00:14.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:00:14.109 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:00:14.110 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:00:14.110 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:00:14.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:00:14.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' -1 2020-09-12T14:00:14.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1598: wait_for_clean: loop=0 2020-09-12T14:00:14.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1599: wait_for_clean: num_active_clean=0 2020-09-12T14:00:14.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:00:14.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:00:14.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:00:14.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:00:14.747 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:00:14.747 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:00:14.747 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:00:14.747 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:00:14.748 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:00:14.748 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:00:15.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:00:15.144 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:00:15.145 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:00:15.145 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:00:15.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:00:15.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:00:15.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:00:15.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:00:15.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:00:15.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:00:15.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:00:15.687 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:00:15.687 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:00:16.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:00:16.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:00:16.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 1 >= 8 )) 2020-09-12T14:00:16.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:00:16.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.2 2020-09-12T14:00:16.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:00:16.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:00:16.421 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:00:16.422 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:00:16.422 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:00:16.422 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:00:16.422 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:00:16.423 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:00:16.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:00:16.817 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:00:16.817 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:00:16.818 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:00:17.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:00:17.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:00:17.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:00:17.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:00:17.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:00:17.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:00:17.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:00:17.351 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:00:17.351 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:00:17.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:00:17.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:00:17.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 2 >= 8 )) 2020-09-12T14:00:17.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:00:17.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.4 2020-09-12T14:00:18.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:00:18.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:00:18.287 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:00:18.287 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:00:18.287 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:00:18.287 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:00:18.287 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:00:18.288 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:00:18.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:00:18.687 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:00:18.687 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:00:18.687 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:00:19.240 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:00:19.240 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:00:19.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:00:19.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:00:19.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:00:19.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:00:19.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:00:19.241 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:00:19.242 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:00:19.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:00:19.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:00:19.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 3 >= 8 )) 2020-09-12T14:00:19.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:00:19.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.8 2020-09-12T14:00:20.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:00:20.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:00:20.578 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:00:20.579 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:00:20.579 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:00:20.579 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:00:20.579 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:00:20.579 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:00:20.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:00:20.987 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:00:20.988 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:00:20.988 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:00:21.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:00:21.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:00:21.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:00:21.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:00:21.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:00:21.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:00:21.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:00:21.536 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:00:21.536 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:00:22.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:00:22.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:00:22.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 4 >= 8 )) 2020-09-12T14:00:22.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:00:22.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 1.6 2020-09-12T14:00:23.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:00:23.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:00:23.683 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:00:23.683 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:00:23.684 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:00:23.684 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:00:23.684 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:00:23.684 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:00:24.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:00:24.100 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:00:24.100 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:00:24.100 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:00:24.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:00:24.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:00:24.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:00:24.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:00:24.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:00:24.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:00:24.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:00:24.637 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:00:24.637 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:00:25.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:00:25.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:00:25.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 5 >= 8 )) 2020-09-12T14:00:25.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:00:25.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 3.2 2020-09-12T14:00:28.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:00:28.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:00:28.381 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:00:28.381 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:00:28.381 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:00:28.381 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:00:28.381 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:00:28.382 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:00:28.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:00:28.795 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:00:28.795 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:00:28.796 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:00:29.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:00:29.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:00:29.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:00:29.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:00:29.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:00:29.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:00:29.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:00:29.328 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:00:29.328 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:00:29.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:00:29.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:00:29.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 6 >= 8 )) 2020-09-12T14:00:29.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:00:29.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 6.4 2020-09-12T14:00:36.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:00:36.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:00:36.269 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:00:36.269 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:00:36.269 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:00:36.270 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:00:36.270 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:00:36.270 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:00:36.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:00:36.676 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:00:36.677 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:00:36.677 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:00:37.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:00:37.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:00:37.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:00:37.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:00:37.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:00:37.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:00:37.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:00:37.213 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:00:37.213 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:00:37.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:00:37.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:00:37.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 7 >= 8 )) 2020-09-12T14:00:37.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:00:37.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 7.3 2020-09-12T14:00:45.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:00:45.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:00:45.051 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:00:45.051 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:00:45.052 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:00:45.052 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:00:45.052 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:00:45.052 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:00:45.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:00:45.462 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:00:45.462 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:00:45.462 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:00:45.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:00:45.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:00:45.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:00:45.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:00:45.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:00:45.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:00:45.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:00:45.993 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:00:45.993 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:00:46.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:00:46.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:00:46.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 8 >= 8 )) 2020-09-12T14:00:46.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: wait_for_clean: ceph report 2020-09-12T14:00:47.053 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:00:47.053 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_fingerprint": "7db1c797-8407-4b23-93e4-602672e04493", 2020-09-12T14:00:47.053 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:00:47.053 INFO:tasks.workunit.client.0.smithi027.stdout: "commit": "f346cc73d15a568d3bec1c99a58953de76f57b85", 2020-09-12T14:00:47.053 INFO:tasks.workunit.client.0.smithi027.stdout: "timestamp": "2020-09-12T14:00:47.051519+0000", 2020-09-12T14:00:47.054 INFO:tasks.workunit.client.0.smithi027.stdout: "tag": "", 2020-09-12T14:00:47.054 INFO:tasks.workunit.client.0.smithi027.stdout: "health": { 2020-09-12T14:00:47.054 INFO:tasks.workunit.client.0.smithi027.stdout: "status": "HEALTH_WARN", 2020-09-12T14:00:47.054 INFO:tasks.workunit.client.0.smithi027.stdout: "checks": { 2020-09-12T14:00:47.054 INFO:tasks.workunit.client.0.smithi027.stdout: "OSDMAP_FLAGS": { 2020-09-12T14:00:47.054 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:00:47.054 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:00:47.055 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "nodown,noout,noin flag(s) set", 2020-09-12T14:00:47.055 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 17 2020-09-12T14:00:47.055 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.055 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [], 2020-09-12T14:00:47.055 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:00:47.055 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.055 INFO:tasks.workunit.client.0.smithi027.stdout: "OSD_DOWN": { 2020-09-12T14:00:47.055 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:00:47.056 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:00:47.056 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "1 osds down", 2020-09-12T14:00:47.056 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 1 2020-09-12T14:00:47.056 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.056 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [ 2020-09-12T14:00:47.056 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.056 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "osd.1 (root=default,host=smithi027) is down" 2020-09-12T14:00:47.057 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.057 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:00:47.057 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:00:47.057 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.057 INFO:tasks.workunit.client.0.smithi027.stdout: "PG_DEGRADED": { 2020-09-12T14:00:47.057 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:00:47.057 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:00:47.057 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 1 pg degraded", 2020-09-12T14:00:47.058 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 1 2020-09-12T14:00:47.058 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.058 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [ 2020-09-12T14:00:47.058 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.058 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2020-09-12T14:00:47.058 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.058 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:00:47.059 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:00:47.059 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.059 INFO:tasks.workunit.client.0.smithi027.stdout: "POOL_APP_NOT_ENABLED": { 2020-09-12T14:00:47.059 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:00:47.059 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:00:47.059 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "1 pool(s) do not have an application enabled", 2020-09-12T14:00:47.059 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 1 2020-09-12T14:00:47.060 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.060 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [ 2020-09-12T14:00:47.060 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.060 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "application not enabled on pool 'test'" 2020-09-12T14:00:47.060 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.060 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.060 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2020-09-12T14:00:47.061 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.061 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:00:47.061 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:00:47.061 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.061 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.061 INFO:tasks.workunit.client.0.smithi027.stdout: "mutes": [] 2020-09-12T14:00:47.061 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.061 INFO:tasks.workunit.client.0.smithi027.stdout: "monmap_first_committed": 1, 2020-09-12T14:00:47.062 INFO:tasks.workunit.client.0.smithi027.stdout: "monmap_last_committed": 1, 2020-09-12T14:00:47.062 INFO:tasks.workunit.client.0.smithi027.stdout: "monmap": { 2020-09-12T14:00:47.062 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 1, 2020-09-12T14:00:47.062 INFO:tasks.workunit.client.0.smithi027.stdout: "fsid": "7e90bad3-7973-4a8c-be65-6326ab931fe8", 2020-09-12T14:00:47.062 INFO:tasks.workunit.client.0.smithi027.stdout: "modified": "2020-09-12T13:58:51.776233Z", 2020-09-12T14:00:47.062 INFO:tasks.workunit.client.0.smithi027.stdout: "created": "2020-09-12T13:58:51.776233Z", 2020-09-12T14:00:47.062 INFO:tasks.workunit.client.0.smithi027.stdout: "min_mon_release": 16, 2020-09-12T14:00:47.063 INFO:tasks.workunit.client.0.smithi027.stdout: "min_mon_release_name": "pacific", 2020-09-12T14:00:47.063 INFO:tasks.workunit.client.0.smithi027.stdout: "features": { 2020-09-12T14:00:47.063 INFO:tasks.workunit.client.0.smithi027.stdout: "persistent": [ 2020-09-12T14:00:47.063 INFO:tasks.workunit.client.0.smithi027.stdout: "kraken", 2020-09-12T14:00:47.063 INFO:tasks.workunit.client.0.smithi027.stdout: "luminous", 2020-09-12T14:00:47.063 INFO:tasks.workunit.client.0.smithi027.stdout: "mimic", 2020-09-12T14:00:47.063 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap-prune", 2020-09-12T14:00:47.063 INFO:tasks.workunit.client.0.smithi027.stdout: "nautilus", 2020-09-12T14:00:47.064 INFO:tasks.workunit.client.0.smithi027.stdout: "octopus", 2020-09-12T14:00:47.064 INFO:tasks.workunit.client.0.smithi027.stdout: "pacific" 2020-09-12T14:00:47.064 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:00:47.064 INFO:tasks.workunit.client.0.smithi027.stdout: "optional": [] 2020-09-12T14:00:47.064 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.064 INFO:tasks.workunit.client.0.smithi027.stdout: "mons": [ 2020-09-12T14:00:47.064 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.064 INFO:tasks.workunit.client.0.smithi027.stdout: "rank": 0, 2020-09-12T14:00:47.065 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "a", 2020-09-12T14:00:47.065 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:00:47.065 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:00:47.065 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.065 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:00:47.065 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:7115", 2020-09-12T14:00:47.065 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 0 2020-09-12T14:00:47.065 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.066 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.066 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.066 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:7115/0", 2020-09-12T14:00:47.066 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:7115/0", 2020-09-12T14:00:47.066 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 0, 2020-09-12T14:00:47.066 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 0 2020-09-12T14:00:47.066 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.067 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.067 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.067 INFO:tasks.workunit.client.0.smithi027.stdout: "quorum": [ 2020-09-12T14:00:47.067 INFO:tasks.workunit.client.0.smithi027.stdout: 0 2020-09-12T14:00:47.067 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:00:47.067 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap": { 2020-09-12T14:00:47.067 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 39, 2020-09-12T14:00:47.067 INFO:tasks.workunit.client.0.smithi027.stdout: "fsid": "7e90bad3-7973-4a8c-be65-6326ab931fe8", 2020-09-12T14:00:47.068 INFO:tasks.workunit.client.0.smithi027.stdout: "created": "2020-09-12T13:58:52.534797+0000", 2020-09-12T14:00:47.068 INFO:tasks.workunit.client.0.smithi027.stdout: "modified": "2020-09-12T14:00:09.995790+0000", 2020-09-12T14:00:47.068 INFO:tasks.workunit.client.0.smithi027.stdout: "last_up_change": "2020-09-12T14:00:08.455706+0000", 2020-09-12T14:00:47.068 INFO:tasks.workunit.client.0.smithi027.stdout: "last_in_change": "2020-09-12T14:00:09.716648+0000", 2020-09-12T14:00:47.068 INFO:tasks.workunit.client.0.smithi027.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-09-12T14:00:47.068 INFO:tasks.workunit.client.0.smithi027.stdout: "flags_num": 5800384, 2020-09-12T14:00:47.068 INFO:tasks.workunit.client.0.smithi027.stdout: "flags_set": [ 2020-09-12T14:00:47.068 INFO:tasks.workunit.client.0.smithi027.stdout: "nodown", 2020-09-12T14:00:47.069 INFO:tasks.workunit.client.0.smithi027.stdout: "noin", 2020-09-12T14:00:47.069 INFO:tasks.workunit.client.0.smithi027.stdout: "noout", 2020-09-12T14:00:47.069 INFO:tasks.workunit.client.0.smithi027.stdout: "pglog_hardlimit", 2020-09-12T14:00:47.069 INFO:tasks.workunit.client.0.smithi027.stdout: "purged_snapdirs", 2020-09-12T14:00:47.069 INFO:tasks.workunit.client.0.smithi027.stdout: "recovery_deletes", 2020-09-12T14:00:47.069 INFO:tasks.workunit.client.0.smithi027.stdout: "sortbitwise" 2020-09-12T14:00:47.069 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:00:47.070 INFO:tasks.workunit.client.0.smithi027.stdout: "crush_version": 7, 2020-09-12T14:00:47.070 INFO:tasks.workunit.client.0.smithi027.stdout: "full_ratio": 0.99000000953674316, 2020-09-12T14:00:47.070 INFO:tasks.workunit.client.0.smithi027.stdout: "backfillfull_ratio": 0.99000000953674316, 2020-09-12T14:00:47.070 INFO:tasks.workunit.client.0.smithi027.stdout: "nearfull_ratio": 0.85000002384185791, 2020-09-12T14:00:47.070 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_snapshot": "", 2020-09-12T14:00:47.070 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_max": 1, 2020-09-12T14:00:47.070 INFO:tasks.workunit.client.0.smithi027.stdout: "max_osd": 3, 2020-09-12T14:00:47.070 INFO:tasks.workunit.client.0.smithi027.stdout: "require_min_compat_client": "luminous", 2020-09-12T14:00:47.071 INFO:tasks.workunit.client.0.smithi027.stdout: "min_compat_client": "jewel", 2020-09-12T14:00:47.071 INFO:tasks.workunit.client.0.smithi027.stdout: "require_osd_release": "pacific", 2020-09-12T14:00:47.071 INFO:tasks.workunit.client.0.smithi027.stdout: "pools": [ 2020-09-12T14:00:47.071 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.071 INFO:tasks.workunit.client.0.smithi027.stdout: "pool": 1, 2020-09-12T14:00:47.071 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_name": "test", 2020-09-12T14:00:47.071 INFO:tasks.workunit.client.0.smithi027.stdout: "create_time": "2020-09-12T13:59:19.085741+0000", 2020-09-12T14:00:47.071 INFO:tasks.workunit.client.0.smithi027.stdout: "flags": 1, 2020-09-12T14:00:47.072 INFO:tasks.workunit.client.0.smithi027.stdout: "flags_names": "hashpspool", 2020-09-12T14:00:47.072 INFO:tasks.workunit.client.0.smithi027.stdout: "type": 1, 2020-09-12T14:00:47.072 INFO:tasks.workunit.client.0.smithi027.stdout: "size": 3, 2020-09-12T14:00:47.072 INFO:tasks.workunit.client.0.smithi027.stdout: "min_size": 2, 2020-09-12T14:00:47.072 INFO:tasks.workunit.client.0.smithi027.stdout: "crush_rule": 0, 2020-09-12T14:00:47.072 INFO:tasks.workunit.client.0.smithi027.stdout: "object_hash": 2, 2020-09-12T14:00:47.072 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_autoscale_mode": "off", 2020-09-12T14:00:47.073 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_num": 1, 2020-09-12T14:00:47.073 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_placement_num": 1, 2020-09-12T14:00:47.073 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_placement_num_target": 1, 2020-09-12T14:00:47.073 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_num_target": 1, 2020-09-12T14:00:47.073 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_num_pending": 1, 2020-09-12T14:00:47.073 INFO:tasks.workunit.client.0.smithi027.stdout: "last_pg_merge_meta": { 2020-09-12T14:00:47.073 INFO:tasks.workunit.client.0.smithi027.stdout: "source_pgid": "0.0", 2020-09-12T14:00:47.073 INFO:tasks.workunit.client.0.smithi027.stdout: "ready_epoch": 0, 2020-09-12T14:00:47.074 INFO:tasks.workunit.client.0.smithi027.stdout: "last_epoch_started": 0, 2020-09-12T14:00:47.074 INFO:tasks.workunit.client.0.smithi027.stdout: "last_epoch_clean": 0, 2020-09-12T14:00:47.074 INFO:tasks.workunit.client.0.smithi027.stdout: "source_version": "0'0", 2020-09-12T14:00:47.074 INFO:tasks.workunit.client.0.smithi027.stdout: "target_version": "0'0" 2020-09-12T14:00:47.074 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.074 INFO:tasks.workunit.client.0.smithi027.stdout: "last_change": "29", 2020-09-12T14:00:47.074 INFO:tasks.workunit.client.0.smithi027.stdout: "last_force_op_resend": "0", 2020-09-12T14:00:47.074 INFO:tasks.workunit.client.0.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2020-09-12T14:00:47.075 INFO:tasks.workunit.client.0.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2020-09-12T14:00:47.075 INFO:tasks.workunit.client.0.smithi027.stdout: "auid": 0, 2020-09-12T14:00:47.075 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_mode": "selfmanaged", 2020-09-12T14:00:47.075 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_seq": 0, 2020-09-12T14:00:47.075 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_epoch": 0, 2020-09-12T14:00:47.075 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_snaps": [], 2020-09-12T14:00:47.075 INFO:tasks.workunit.client.0.smithi027.stdout: "removed_snaps": "[]", 2020-09-12T14:00:47.076 INFO:tasks.workunit.client.0.smithi027.stdout: "quota_max_bytes": 0, 2020-09-12T14:00:47.076 INFO:tasks.workunit.client.0.smithi027.stdout: "quota_max_objects": 0, 2020-09-12T14:00:47.076 INFO:tasks.workunit.client.0.smithi027.stdout: "tiers": [], 2020-09-12T14:00:47.076 INFO:tasks.workunit.client.0.smithi027.stdout: "tier_of": -1, 2020-09-12T14:00:47.076 INFO:tasks.workunit.client.0.smithi027.stdout: "read_tier": -1, 2020-09-12T14:00:47.076 INFO:tasks.workunit.client.0.smithi027.stdout: "write_tier": -1, 2020-09-12T14:00:47.076 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_mode": "none", 2020-09-12T14:00:47.076 INFO:tasks.workunit.client.0.smithi027.stdout: "target_max_bytes": 0, 2020-09-12T14:00:47.077 INFO:tasks.workunit.client.0.smithi027.stdout: "target_max_objects": 0, 2020-09-12T14:00:47.077 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-09-12T14:00:47.077 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-09-12T14:00:47.077 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2020-09-12T14:00:47.077 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_min_flush_age": 0, 2020-09-12T14:00:47.077 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_min_evict_age": 0, 2020-09-12T14:00:47.077 INFO:tasks.workunit.client.0.smithi027.stdout: "erasure_code_profile": "", 2020-09-12T14:00:47.077 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_params": { 2020-09-12T14:00:47.078 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "none" 2020-09-12T14:00:47.078 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.078 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_period": 0, 2020-09-12T14:00:47.078 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_count": 0, 2020-09-12T14:00:47.078 INFO:tasks.workunit.client.0.smithi027.stdout: "use_gmt_hitset": true, 2020-09-12T14:00:47.078 INFO:tasks.workunit.client.0.smithi027.stdout: "min_read_recency_for_promote": 0, 2020-09-12T14:00:47.078 INFO:tasks.workunit.client.0.smithi027.stdout: "min_write_recency_for_promote": 0, 2020-09-12T14:00:47.079 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2020-09-12T14:00:47.079 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_search_last_n": 0, 2020-09-12T14:00:47.079 INFO:tasks.workunit.client.0.smithi027.stdout: "grade_table": [], 2020-09-12T14:00:47.079 INFO:tasks.workunit.client.0.smithi027.stdout: "stripe_width": 0, 2020-09-12T14:00:47.079 INFO:tasks.workunit.client.0.smithi027.stdout: "expected_num_objects": 0, 2020-09-12T14:00:47.079 INFO:tasks.workunit.client.0.smithi027.stdout: "fast_read": false, 2020-09-12T14:00:47.079 INFO:tasks.workunit.client.0.smithi027.stdout: "options": {}, 2020-09-12T14:00:47.079 INFO:tasks.workunit.client.0.smithi027.stdout: "application_metadata": {} 2020-09-12T14:00:47.080 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.080 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:00:47.080 INFO:tasks.workunit.client.0.smithi027.stdout: "osds": [ 2020-09-12T14:00:47.080 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.080 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:00:47.080 INFO:tasks.workunit.client.0.smithi027.stdout: "uuid": "6e938613-9665-406c-b148-ce5bfd24c111", 2020-09-12T14:00:47.080 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 1, 2020-09-12T14:00:47.080 INFO:tasks.workunit.client.0.smithi027.stdout: "in": 1, 2020-09-12T14:00:47.081 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 1, 2020-09-12T14:00:47.081 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_affinity": 1, 2020-09-12T14:00:47.081 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_begin": 5, 2020-09-12T14:00:47.081 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_end": 29, 2020-09-12T14:00:47.081 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 35, 2020-09-12T14:00:47.081 INFO:tasks.workunit.client.0.smithi027.stdout: "up_thru": 38, 2020-09-12T14:00:47.081 INFO:tasks.workunit.client.0.smithi027.stdout: "down_at": 30, 2020-09-12T14:00:47.081 INFO:tasks.workunit.client.0.smithi027.stdout: "lost_at": 0, 2020-09-12T14:00:47.082 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:00:47.082 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:00:47.082 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.082 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:00:47.082 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6802", 2020-09-12T14:00:47.082 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37498 2020-09-12T14:00:47.082 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.083 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.083 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:00:47.083 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6803", 2020-09-12T14:00:47.083 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37498 2020-09-12T14:00:47.083 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.083 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.083 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.083 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addrs": { 2020-09-12T14:00:47.084 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:00:47.084 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.084 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:00:47.084 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6804", 2020-09-12T14:00:47.084 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37498 2020-09-12T14:00:47.084 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.084 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.085 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:00:47.085 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6805", 2020-09-12T14:00:47.085 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37498 2020-09-12T14:00:47.085 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.085 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.085 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.085 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addrs": { 2020-09-12T14:00:47.085 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:00:47.086 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.086 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:00:47.086 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6808", 2020-09-12T14:00:47.086 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37498 2020-09-12T14:00:47.086 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.086 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.086 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:00:47.086 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6809", 2020-09-12T14:00:47.087 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37498 2020-09-12T14:00:47.087 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.087 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.087 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.087 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addrs": { 2020-09-12T14:00:47.087 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:00:47.087 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.088 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:00:47.088 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6806", 2020-09-12T14:00:47.088 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37498 2020-09-12T14:00:47.088 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.088 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.088 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:00:47.088 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6807", 2020-09-12T14:00:47.088 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37498 2020-09-12T14:00:47.089 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.089 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.089 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.089 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:6803/37498", 2020-09-12T14:00:47.089 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addr": "127.0.0.1:6805/37498", 2020-09-12T14:00:47.089 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addr": "127.0.0.1:6809/37498", 2020-09-12T14:00:47.089 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addr": "127.0.0.1:6807/37498", 2020-09-12T14:00:47.089 INFO:tasks.workunit.client.0.smithi027.stdout: "state": [ 2020-09-12T14:00:47.090 INFO:tasks.workunit.client.0.smithi027.stdout: "exists", 2020-09-12T14:00:47.090 INFO:tasks.workunit.client.0.smithi027.stdout: "up" 2020-09-12T14:00:47.090 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.090 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.090 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.090 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:00:47.090 INFO:tasks.workunit.client.0.smithi027.stdout: "uuid": "d92b7341-d129-4e21-85d9-c431d9f9298f", 2020-09-12T14:00:47.090 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 0, 2020-09-12T14:00:47.091 INFO:tasks.workunit.client.0.smithi027.stdout: "in": 1, 2020-09-12T14:00:47.091 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 1, 2020-09-12T14:00:47.091 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_affinity": 1, 2020-09-12T14:00:47.091 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_begin": 0, 2020-09-12T14:00:47.091 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_end": 0, 2020-09-12T14:00:47.091 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 10, 2020-09-12T14:00:47.091 INFO:tasks.workunit.client.0.smithi027.stdout: "up_thru": 23, 2020-09-12T14:00:47.092 INFO:tasks.workunit.client.0.smithi027.stdout: "down_at": 31, 2020-09-12T14:00:47.092 INFO:tasks.workunit.client.0.smithi027.stdout: "lost_at": 0, 2020-09-12T14:00:47.092 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:00:47.092 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:00:47.092 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.092 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:00:47.092 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6810", 2020-09-12T14:00:47.092 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 32590 2020-09-12T14:00:47.093 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.093 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.093 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:00:47.093 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6811", 2020-09-12T14:00:47.093 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 32590 2020-09-12T14:00:47.093 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.093 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.093 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.094 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addrs": { 2020-09-12T14:00:47.094 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:00:47.094 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.094 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:00:47.094 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6812", 2020-09-12T14:00:47.094 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 32590 2020-09-12T14:00:47.094 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.095 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.095 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:00:47.095 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6813", 2020-09-12T14:00:47.095 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 32590 2020-09-12T14:00:47.095 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.095 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.095 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.095 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addrs": { 2020-09-12T14:00:47.096 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:00:47.096 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.096 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:00:47.096 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6816", 2020-09-12T14:00:47.096 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 32590 2020-09-12T14:00:47.096 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.096 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.096 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:00:47.097 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6817", 2020-09-12T14:00:47.097 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 32590 2020-09-12T14:00:47.097 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.097 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.097 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.097 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addrs": { 2020-09-12T14:00:47.097 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:00:47.097 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.098 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:00:47.098 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6814", 2020-09-12T14:00:47.098 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 32590 2020-09-12T14:00:47.098 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.098 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.098 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:00:47.098 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6815", 2020-09-12T14:00:47.098 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 32590 2020-09-12T14:00:47.099 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.099 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.099 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.099 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:6811/32590", 2020-09-12T14:00:47.099 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addr": "127.0.0.1:6813/32590", 2020-09-12T14:00:47.099 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addr": "127.0.0.1:6817/32590", 2020-09-12T14:00:47.099 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addr": "127.0.0.1:6815/32590", 2020-09-12T14:00:47.100 INFO:tasks.workunit.client.0.smithi027.stdout: "state": [ 2020-09-12T14:00:47.100 INFO:tasks.workunit.client.0.smithi027.stdout: "exists" 2020-09-12T14:00:47.100 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.100 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.100 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.100 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:00:47.100 INFO:tasks.workunit.client.0.smithi027.stdout: "uuid": "1b25b33b-61e6-4d39-ab58-fee24e94b77f", 2020-09-12T14:00:47.100 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 1, 2020-09-12T14:00:47.101 INFO:tasks.workunit.client.0.smithi027.stdout: "in": 1, 2020-09-12T14:00:47.101 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 1, 2020-09-12T14:00:47.101 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_affinity": 1, 2020-09-12T14:00:47.101 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_begin": 15, 2020-09-12T14:00:47.101 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_end": 29, 2020-09-12T14:00:47.101 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 36, 2020-09-12T14:00:47.101 INFO:tasks.workunit.client.0.smithi027.stdout: "up_thru": 0, 2020-09-12T14:00:47.101 INFO:tasks.workunit.client.0.smithi027.stdout: "down_at": 32, 2020-09-12T14:00:47.102 INFO:tasks.workunit.client.0.smithi027.stdout: "lost_at": 0, 2020-09-12T14:00:47.102 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:00:47.102 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:00:47.102 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.102 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:00:47.102 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6810", 2020-09-12T14:00:47.102 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37897 2020-09-12T14:00:47.102 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.103 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.103 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:00:47.103 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6811", 2020-09-12T14:00:47.103 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37897 2020-09-12T14:00:47.103 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.103 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.103 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.104 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addrs": { 2020-09-12T14:00:47.104 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:00:47.104 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.104 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:00:47.104 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6812", 2020-09-12T14:00:47.104 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37897 2020-09-12T14:00:47.104 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.104 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.105 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:00:47.105 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6813", 2020-09-12T14:00:47.105 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37897 2020-09-12T14:00:47.105 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.105 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.105 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.105 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addrs": { 2020-09-12T14:00:47.105 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:00:47.106 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.106 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:00:47.106 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6816", 2020-09-12T14:00:47.106 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37897 2020-09-12T14:00:47.106 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.106 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.106 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:00:47.107 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6817", 2020-09-12T14:00:47.107 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37897 2020-09-12T14:00:47.107 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.107 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.107 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.107 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addrs": { 2020-09-12T14:00:47.107 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:00:47.107 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.108 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:00:47.108 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6814", 2020-09-12T14:00:47.108 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37897 2020-09-12T14:00:47.108 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.108 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.108 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:00:47.108 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6815", 2020-09-12T14:00:47.108 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37897 2020-09-12T14:00:47.109 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.109 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.109 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.109 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:6811/37897", 2020-09-12T14:00:47.109 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addr": "127.0.0.1:6813/37897", 2020-09-12T14:00:47.109 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addr": "127.0.0.1:6817/37897", 2020-09-12T14:00:47.109 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addr": "127.0.0.1:6815/37897", 2020-09-12T14:00:47.109 INFO:tasks.workunit.client.0.smithi027.stdout: "state": [ 2020-09-12T14:00:47.110 INFO:tasks.workunit.client.0.smithi027.stdout: "exists", 2020-09-12T14:00:47.110 INFO:tasks.workunit.client.0.smithi027.stdout: "up" 2020-09-12T14:00:47.110 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.110 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.110 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:00:47.110 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_xinfo": [ 2020-09-12T14:00:47.110 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.111 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:00:47.111 INFO:tasks.workunit.client.0.smithi027.stdout: "down_stamp": "2020-09-12T14:00:01.267275+0000", 2020-09-12T14:00:47.111 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_probability": 0, 2020-09-12T14:00:47.111 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_interval": 0, 2020-09-12T14:00:47.111 INFO:tasks.workunit.client.0.smithi027.stdout: "features": 4540138292841938943, 2020-09-12T14:00:47.111 INFO:tasks.workunit.client.0.smithi027.stdout: "old_weight": 0, 2020-09-12T14:00:47.111 INFO:tasks.workunit.client.0.smithi027.stdout: "last_purged_snaps_scrub": "2020-09-12T13:59:00.980797+0000", 2020-09-12T14:00:47.111 INFO:tasks.workunit.client.0.smithi027.stdout: "dead_epoch": 0 2020-09-12T14:00:47.112 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.112 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.112 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:00:47.112 INFO:tasks.workunit.client.0.smithi027.stdout: "down_stamp": "2020-09-12T14:00:01.870221+0000", 2020-09-12T14:00:47.112 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_probability": 0, 2020-09-12T14:00:47.112 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_interval": 0, 2020-09-12T14:00:47.112 INFO:tasks.workunit.client.0.smithi027.stdout: "features": 4540138292841938943, 2020-09-12T14:00:47.112 INFO:tasks.workunit.client.0.smithi027.stdout: "old_weight": 0, 2020-09-12T14:00:47.113 INFO:tasks.workunit.client.0.smithi027.stdout: "last_purged_snaps_scrub": "2020-09-12T13:59:08.852599+0000", 2020-09-12T14:00:47.113 INFO:tasks.workunit.client.0.smithi027.stdout: "dead_epoch": 0 2020-09-12T14:00:47.113 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.113 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.113 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:00:47.113 INFO:tasks.workunit.client.0.smithi027.stdout: "down_stamp": "2020-09-12T14:00:02.440346+0000", 2020-09-12T14:00:47.113 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_probability": 0, 2020-09-12T14:00:47.114 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_interval": 0, 2020-09-12T14:00:47.114 INFO:tasks.workunit.client.0.smithi027.stdout: "features": 4540138292841938943, 2020-09-12T14:00:47.114 INFO:tasks.workunit.client.0.smithi027.stdout: "old_weight": 0, 2020-09-12T14:00:47.114 INFO:tasks.workunit.client.0.smithi027.stdout: "last_purged_snaps_scrub": "2020-09-12T13:59:16.540209+0000", 2020-09-12T14:00:47.114 INFO:tasks.workunit.client.0.smithi027.stdout: "dead_epoch": 0 2020-09-12T14:00:47.114 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.114 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:00:47.114 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_upmap": [], 2020-09-12T14:00:47.115 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_upmap_items": [], 2020-09-12T14:00:47.115 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_temp": [], 2020-09-12T14:00:47.115 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_temp": [], 2020-09-12T14:00:47.115 INFO:tasks.workunit.client.0.smithi027.stdout: "blocklist": {}, 2020-09-12T14:00:47.115 INFO:tasks.workunit.client.0.smithi027.stdout: "erasure_code_profiles": { 2020-09-12T14:00:47.115 INFO:tasks.workunit.client.0.smithi027.stdout: "default": { 2020-09-12T14:00:47.115 INFO:tasks.workunit.client.0.smithi027.stdout: "k": "2", 2020-09-12T14:00:47.116 INFO:tasks.workunit.client.0.smithi027.stdout: "m": "2", 2020-09-12T14:00:47.116 INFO:tasks.workunit.client.0.smithi027.stdout: "plugin": "jerasure", 2020-09-12T14:00:47.116 INFO:tasks.workunit.client.0.smithi027.stdout: "technique": "reed_sol_van" 2020-09-12T14:00:47.116 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.116 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.116 INFO:tasks.workunit.client.0.smithi027.stdout: "removed_snaps_queue": [], 2020-09-12T14:00:47.116 INFO:tasks.workunit.client.0.smithi027.stdout: "new_removed_snaps": [], 2020-09-12T14:00:47.116 INFO:tasks.workunit.client.0.smithi027.stdout: "new_purged_snaps": [], 2020-09-12T14:00:47.117 INFO:tasks.workunit.client.0.smithi027.stdout: "crush_node_flags": {}, 2020-09-12T14:00:47.117 INFO:tasks.workunit.client.0.smithi027.stdout: "device_class_flags": {} 2020-09-12T14:00:47.117 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.117 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_metadata": [ 2020-09-12T14:00:47.117 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.117 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:00:47.117 INFO:tasks.workunit.client.0.smithi027.stdout: "arch": "x86_64", 2020-09-12T14:00:47.117 INFO:tasks.workunit.client.0.smithi027.stdout: "back_addr": "[v2:127.0.0.1:6804/37498,v1:127.0.0.1:6805/37498]", 2020-09-12T14:00:47.118 INFO:tasks.workunit.client.0.smithi027.stdout: "back_iface": "", 2020-09-12T14:00:47.118 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs": "1", 2020-09-12T14:00:47.118 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_db": "0", 2020-09-12T14:00:47.118 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_wal": "0", 2020-09-12T14:00:47.118 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_single_shared_device": "1", 2020-09-12T14:00:47.118 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_access_mode": "file", 2020-09-12T14:00:47.118 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_block_size": "4096", 2020-09-12T14:00:47.119 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_devices": "sda", 2020-09-12T14:00:47.119 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_driver": "KernelDevice", 2020-09-12T14:00:47.119 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2020-09-12T14:00:47.119 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_rotational": "1", 2020-09-12T14:00:47.119 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_size": "107374182400", 2020-09-12T14:00:47.119 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_support_discard": "0", 2020-09-12T14:00:47.119 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_type": "hdd", 2020-09-12T14:00:47.119 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_release": "pacific", 2020-09-12T14:00:47.120 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version": "ceph version 16.0.0-5485-gf346cc73d15 (f346cc73d15a568d3bec1c99a58953de76f57b85) pacific (dev)", 2020-09-12T14:00:47.120 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version_short": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:00:47.120 INFO:tasks.workunit.client.0.smithi027.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2020-09-12T14:00:47.120 INFO:tasks.workunit.client.0.smithi027.stdout: "default_device_class": "hdd", 2020-09-12T14:00:47.120 INFO:tasks.workunit.client.0.smithi027.stdout: "device_ids": "sda=ST1000NM0033-9ZM173_Z1W4H3N7", 2020-09-12T14:00:47.120 INFO:tasks.workunit.client.0.smithi027.stdout: "device_paths": "sda=/dev/disk/by-path/pci-0000:00:1f.2-ata-1", 2020-09-12T14:00:47.120 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": "sda", 2020-09-12T14:00:47.120 INFO:tasks.workunit.client.0.smithi027.stdout: "distro": "centos", 2020-09-12T14:00:47.121 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_description": "CentOS Linux 8 (Core)", 2020-09-12T14:00:47.121 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_version": "8", 2020-09-12T14:00:47.121 INFO:tasks.workunit.client.0.smithi027.stdout: "front_addr": "[v2:127.0.0.1:6802/37498,v1:127.0.0.1:6803/37498]", 2020-09-12T14:00:47.121 INFO:tasks.workunit.client.0.smithi027.stdout: "front_iface": "", 2020-09-12T14:00:47.121 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/37498,v1:127.0.0.1:6809/37498]", 2020-09-12T14:00:47.121 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/37498,v1:127.0.0.1:6807/37498]", 2020-09-12T14:00:47.121 INFO:tasks.workunit.client.0.smithi027.stdout: "hostname": "smithi027", 2020-09-12T14:00:47.121 INFO:tasks.workunit.client.0.smithi027.stdout: "journal_rotational": "1", 2020-09-12T14:00:47.122 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_description": "#1 SMP Sun Jul 26 03:54:29 UTC 2020", 2020-09-12T14:00:47.122 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_version": "4.18.0-193.14.2.el8_2.x86_64", 2020-09-12T14:00:47.122 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_swap_kb": "0", 2020-09-12T14:00:47.122 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_total_kb": "32659336", 2020-09-12T14:00:47.122 INFO:tasks.workunit.client.0.smithi027.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2020-09-12T14:00:47.122 INFO:tasks.workunit.client.0.smithi027.stdout: "objectstore_numa_unknown_devices": "sda", 2020-09-12T14:00:47.122 INFO:tasks.workunit.client.0.smithi027.stdout: "os": "Linux", 2020-09-12T14:00:47.123 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_data": "td/divergent-priors/0", 2020-09-12T14:00:47.123 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_objectstore": "bluestore", 2020-09-12T14:00:47.123 INFO:tasks.workunit.client.0.smithi027.stdout: "osdspec_affinity": "", 2020-09-12T14:00:47.123 INFO:tasks.workunit.client.0.smithi027.stdout: "rotational": "1" 2020-09-12T14:00:47.123 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.123 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.123 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:00:47.123 INFO:tasks.workunit.client.0.smithi027.stdout: "arch": "x86_64", 2020-09-12T14:00:47.124 INFO:tasks.workunit.client.0.smithi027.stdout: "back_addr": "[v2:127.0.0.1:6812/32590,v1:127.0.0.1:6813/32590]", 2020-09-12T14:00:47.124 INFO:tasks.workunit.client.0.smithi027.stdout: "back_iface": "", 2020-09-12T14:00:47.124 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs": "1", 2020-09-12T14:00:47.124 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_db": "0", 2020-09-12T14:00:47.124 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_wal": "0", 2020-09-12T14:00:47.124 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_single_shared_device": "1", 2020-09-12T14:00:47.124 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_access_mode": "file", 2020-09-12T14:00:47.124 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_block_size": "4096", 2020-09-12T14:00:47.125 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_devices": "sda", 2020-09-12T14:00:47.125 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_driver": "KernelDevice", 2020-09-12T14:00:47.125 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2020-09-12T14:00:47.125 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_rotational": "1", 2020-09-12T14:00:47.125 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_size": "107374182400", 2020-09-12T14:00:47.125 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_support_discard": "0", 2020-09-12T14:00:47.125 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_type": "hdd", 2020-09-12T14:00:47.125 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_release": "pacific", 2020-09-12T14:00:47.126 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version": "ceph version 16.0.0-5485-gf346cc73d15 (f346cc73d15a568d3bec1c99a58953de76f57b85) pacific (dev)", 2020-09-12T14:00:47.126 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version_short": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:00:47.126 INFO:tasks.workunit.client.0.smithi027.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2020-09-12T14:00:47.126 INFO:tasks.workunit.client.0.smithi027.stdout: "default_device_class": "hdd", 2020-09-12T14:00:47.126 INFO:tasks.workunit.client.0.smithi027.stdout: "device_ids": "sda=ST1000NM0033-9ZM173_Z1W4H3N7", 2020-09-12T14:00:47.126 INFO:tasks.workunit.client.0.smithi027.stdout: "device_paths": "sda=/dev/disk/by-path/pci-0000:00:1f.2-ata-1", 2020-09-12T14:00:47.126 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": "sda", 2020-09-12T14:00:47.127 INFO:tasks.workunit.client.0.smithi027.stdout: "distro": "centos", 2020-09-12T14:00:47.127 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_description": "CentOS Linux 8 (Core)", 2020-09-12T14:00:47.127 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_version": "8", 2020-09-12T14:00:47.127 INFO:tasks.workunit.client.0.smithi027.stdout: "front_addr": "[v2:127.0.0.1:6810/32590,v1:127.0.0.1:6811/32590]", 2020-09-12T14:00:47.127 INFO:tasks.workunit.client.0.smithi027.stdout: "front_iface": "", 2020-09-12T14:00:47.127 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/32590,v1:127.0.0.1:6817/32590]", 2020-09-12T14:00:47.127 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/32590,v1:127.0.0.1:6815/32590]", 2020-09-12T14:00:47.127 INFO:tasks.workunit.client.0.smithi027.stdout: "hostname": "smithi027", 2020-09-12T14:00:47.128 INFO:tasks.workunit.client.0.smithi027.stdout: "journal_rotational": "1", 2020-09-12T14:00:47.128 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_description": "#1 SMP Sun Jul 26 03:54:29 UTC 2020", 2020-09-12T14:00:47.128 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_version": "4.18.0-193.14.2.el8_2.x86_64", 2020-09-12T14:00:47.128 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_swap_kb": "0", 2020-09-12T14:00:47.128 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_total_kb": "32659336", 2020-09-12T14:00:47.128 INFO:tasks.workunit.client.0.smithi027.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2020-09-12T14:00:47.128 INFO:tasks.workunit.client.0.smithi027.stdout: "objectstore_numa_unknown_devices": "sda", 2020-09-12T14:00:47.128 INFO:tasks.workunit.client.0.smithi027.stdout: "os": "Linux", 2020-09-12T14:00:47.129 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_data": "td/divergent-priors/1", 2020-09-12T14:00:47.129 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_objectstore": "bluestore", 2020-09-12T14:00:47.129 INFO:tasks.workunit.client.0.smithi027.stdout: "osdspec_affinity": "", 2020-09-12T14:00:47.129 INFO:tasks.workunit.client.0.smithi027.stdout: "rotational": "1" 2020-09-12T14:00:47.129 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.129 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.129 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:00:47.129 INFO:tasks.workunit.client.0.smithi027.stdout: "arch": "x86_64", 2020-09-12T14:00:47.130 INFO:tasks.workunit.client.0.smithi027.stdout: "back_addr": "[v2:127.0.0.1:6812/37897,v1:127.0.0.1:6813/37897]", 2020-09-12T14:00:47.130 INFO:tasks.workunit.client.0.smithi027.stdout: "back_iface": "", 2020-09-12T14:00:47.130 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs": "1", 2020-09-12T14:00:47.130 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_db": "0", 2020-09-12T14:00:47.130 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_wal": "0", 2020-09-12T14:00:47.130 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_single_shared_device": "1", 2020-09-12T14:00:47.130 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_access_mode": "file", 2020-09-12T14:00:47.131 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_block_size": "4096", 2020-09-12T14:00:47.131 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_devices": "sda", 2020-09-12T14:00:47.131 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_driver": "KernelDevice", 2020-09-12T14:00:47.131 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2020-09-12T14:00:47.131 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_rotational": "1", 2020-09-12T14:00:47.131 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_size": "107374182400", 2020-09-12T14:00:47.131 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_support_discard": "0", 2020-09-12T14:00:47.131 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_type": "hdd", 2020-09-12T14:00:47.132 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_release": "pacific", 2020-09-12T14:00:47.132 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version": "ceph version 16.0.0-5485-gf346cc73d15 (f346cc73d15a568d3bec1c99a58953de76f57b85) pacific (dev)", 2020-09-12T14:00:47.132 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version_short": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:00:47.132 INFO:tasks.workunit.client.0.smithi027.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2020-09-12T14:00:47.132 INFO:tasks.workunit.client.0.smithi027.stdout: "default_device_class": "hdd", 2020-09-12T14:00:47.132 INFO:tasks.workunit.client.0.smithi027.stdout: "device_ids": "sda=ST1000NM0033-9ZM173_Z1W4H3N7", 2020-09-12T14:00:47.132 INFO:tasks.workunit.client.0.smithi027.stdout: "device_paths": "sda=/dev/disk/by-path/pci-0000:00:1f.2-ata-1", 2020-09-12T14:00:47.132 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": "sda", 2020-09-12T14:00:47.133 INFO:tasks.workunit.client.0.smithi027.stdout: "distro": "centos", 2020-09-12T14:00:47.133 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_description": "CentOS Linux 8 (Core)", 2020-09-12T14:00:47.133 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_version": "8", 2020-09-12T14:00:47.133 INFO:tasks.workunit.client.0.smithi027.stdout: "front_addr": "[v2:127.0.0.1:6810/37897,v1:127.0.0.1:6811/37897]", 2020-09-12T14:00:47.133 INFO:tasks.workunit.client.0.smithi027.stdout: "front_iface": "", 2020-09-12T14:00:47.133 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/37897,v1:127.0.0.1:6817/37897]", 2020-09-12T14:00:47.133 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/37897,v1:127.0.0.1:6815/37897]", 2020-09-12T14:00:47.133 INFO:tasks.workunit.client.0.smithi027.stdout: "hostname": "smithi027", 2020-09-12T14:00:47.134 INFO:tasks.workunit.client.0.smithi027.stdout: "journal_rotational": "1", 2020-09-12T14:00:47.134 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_description": "#1 SMP Sun Jul 26 03:54:29 UTC 2020", 2020-09-12T14:00:47.134 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_version": "4.18.0-193.14.2.el8_2.x86_64", 2020-09-12T14:00:47.134 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_swap_kb": "0", 2020-09-12T14:00:47.134 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_total_kb": "32659336", 2020-09-12T14:00:47.134 INFO:tasks.workunit.client.0.smithi027.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2020-09-12T14:00:47.134 INFO:tasks.workunit.client.0.smithi027.stdout: "objectstore_numa_unknown_devices": "sda", 2020-09-12T14:00:47.135 INFO:tasks.workunit.client.0.smithi027.stdout: "os": "Linux", 2020-09-12T14:00:47.135 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_data": "td/divergent-priors/2", 2020-09-12T14:00:47.135 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_objectstore": "bluestore", 2020-09-12T14:00:47.135 INFO:tasks.workunit.client.0.smithi027.stdout: "osdspec_affinity": "", 2020-09-12T14:00:47.135 INFO:tasks.workunit.client.0.smithi027.stdout: "rotational": "1" 2020-09-12T14:00:47.135 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.135 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:00:47.135 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap_clean_epochs": { 2020-09-12T14:00:47.136 INFO:tasks.workunit.client.0.smithi027.stdout: "min_last_epoch_clean": 0, 2020-09-12T14:00:47.136 INFO:tasks.workunit.client.0.smithi027.stdout: "last_epoch_clean": { 2020-09-12T14:00:47.136 INFO:tasks.workunit.client.0.smithi027.stdout: "per_pool": [] 2020-09-12T14:00:47.136 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.136 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_epochs": [ 2020-09-12T14:00:47.136 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.136 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:00:47.136 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 35 2020-09-12T14:00:47.137 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.137 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.137 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:00:47.137 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 10 2020-09-12T14:00:47.137 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.137 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.137 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:00:47.137 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 37 2020-09-12T14:00:47.138 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.138 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.138 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.138 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap_first_committed": 1, 2020-09-12T14:00:47.138 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap_last_committed": 39, 2020-09-12T14:00:47.138 INFO:tasks.workunit.client.0.smithi027.stdout: "crushmap": { 2020-09-12T14:00:47.138 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": [ 2020-09-12T14:00:47.138 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.139 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:00:47.139 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd.0", 2020-09-12T14:00:47.139 INFO:tasks.workunit.client.0.smithi027.stdout: "class": "hdd" 2020-09-12T14:00:47.139 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.139 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.139 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:00:47.139 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd.1", 2020-09-12T14:00:47.140 INFO:tasks.workunit.client.0.smithi027.stdout: "class": "hdd" 2020-09-12T14:00:47.140 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.140 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.140 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:00:47.140 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd.2", 2020-09-12T14:00:47.140 INFO:tasks.workunit.client.0.smithi027.stdout: "class": "hdd" 2020-09-12T14:00:47.140 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.140 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:00:47.141 INFO:tasks.workunit.client.0.smithi027.stdout: "types": [ 2020-09-12T14:00:47.141 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.141 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 0, 2020-09-12T14:00:47.141 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd" 2020-09-12T14:00:47.141 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.141 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.141 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 1, 2020-09-12T14:00:47.141 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "host" 2020-09-12T14:00:47.142 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.142 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.142 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 2, 2020-09-12T14:00:47.142 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "chassis" 2020-09-12T14:00:47.142 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.142 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.142 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 3, 2020-09-12T14:00:47.142 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "rack" 2020-09-12T14:00:47.143 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.143 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.143 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 4, 2020-09-12T14:00:47.143 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "row" 2020-09-12T14:00:47.143 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.143 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.143 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 5, 2020-09-12T14:00:47.143 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "pdu" 2020-09-12T14:00:47.144 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.144 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.144 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 6, 2020-09-12T14:00:47.144 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "pod" 2020-09-12T14:00:47.144 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.144 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.144 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 7, 2020-09-12T14:00:47.145 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "room" 2020-09-12T14:00:47.145 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.145 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.145 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 8, 2020-09-12T14:00:47.145 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "datacenter" 2020-09-12T14:00:47.145 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.145 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.145 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 9, 2020-09-12T14:00:47.146 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "zone" 2020-09-12T14:00:47.146 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.146 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.146 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 10, 2020-09-12T14:00:47.146 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "region" 2020-09-12T14:00:47.146 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.146 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.146 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 11, 2020-09-12T14:00:47.147 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "root" 2020-09-12T14:00:47.147 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.147 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:00:47.147 INFO:tasks.workunit.client.0.smithi027.stdout: "buckets": [ 2020-09-12T14:00:47.147 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.147 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -1, 2020-09-12T14:00:47.147 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "default", 2020-09-12T14:00:47.147 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 11, 2020-09-12T14:00:47.148 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "root", 2020-09-12T14:00:47.148 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:00:47.148 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:00:47.148 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:00:47.148 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:00:47.148 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.148 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -3, 2020-09-12T14:00:47.148 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:00:47.149 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:00:47.149 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.149 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.149 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.149 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.149 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -2, 2020-09-12T14:00:47.149 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "default~hdd", 2020-09-12T14:00:47.149 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 11, 2020-09-12T14:00:47.150 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "root", 2020-09-12T14:00:47.150 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:00:47.150 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:00:47.150 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:00:47.150 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:00:47.150 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.150 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -4, 2020-09-12T14:00:47.151 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:00:47.151 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:00:47.151 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.151 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.151 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.151 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.151 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -3, 2020-09-12T14:00:47.151 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "smithi027", 2020-09-12T14:00:47.152 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 1, 2020-09-12T14:00:47.152 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "host", 2020-09-12T14:00:47.152 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:00:47.152 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:00:47.152 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:00:47.152 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:00:47.152 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.152 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:00:47.153 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:00:47.153 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:00:47.153 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.153 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.153 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:00:47.153 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:00:47.153 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 1 2020-09-12T14:00:47.153 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.154 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.154 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:00:47.154 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:00:47.154 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 2 2020-09-12T14:00:47.154 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.154 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.154 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.154 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.155 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -4, 2020-09-12T14:00:47.155 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "smithi027~hdd", 2020-09-12T14:00:47.155 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 1, 2020-09-12T14:00:47.155 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "host", 2020-09-12T14:00:47.155 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:00:47.155 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:00:47.155 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:00:47.156 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:00:47.156 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.156 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:00:47.156 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:00:47.156 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:00:47.156 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.156 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.156 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:00:47.157 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:00:47.157 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 1 2020-09-12T14:00:47.157 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.157 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.157 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:00:47.157 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:00:47.157 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 2 2020-09-12T14:00:47.157 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.158 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.158 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.158 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:00:47.158 INFO:tasks.workunit.client.0.smithi027.stdout: "rules": [ 2020-09-12T14:00:47.158 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.158 INFO:tasks.workunit.client.0.smithi027.stdout: "rule_id": 0, 2020-09-12T14:00:47.158 INFO:tasks.workunit.client.0.smithi027.stdout: "rule_name": "replicated_rule", 2020-09-12T14:00:47.158 INFO:tasks.workunit.client.0.smithi027.stdout: "ruleset": 0, 2020-09-12T14:00:47.159 INFO:tasks.workunit.client.0.smithi027.stdout: "type": 1, 2020-09-12T14:00:47.159 INFO:tasks.workunit.client.0.smithi027.stdout: "min_size": 1, 2020-09-12T14:00:47.159 INFO:tasks.workunit.client.0.smithi027.stdout: "max_size": 10, 2020-09-12T14:00:47.159 INFO:tasks.workunit.client.0.smithi027.stdout: "steps": [ 2020-09-12T14:00:47.159 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.159 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "take", 2020-09-12T14:00:47.159 INFO:tasks.workunit.client.0.smithi027.stdout: "item": -1, 2020-09-12T14:00:47.159 INFO:tasks.workunit.client.0.smithi027.stdout: "item_name": "default" 2020-09-12T14:00:47.160 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.160 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.160 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "choose_firstn", 2020-09-12T14:00:47.160 INFO:tasks.workunit.client.0.smithi027.stdout: "num": 0, 2020-09-12T14:00:47.160 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "osd" 2020-09-12T14:00:47.160 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.160 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.161 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "emit" 2020-09-12T14:00:47.161 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.161 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:00:47.161 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.161 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:00:47.161 INFO:tasks.workunit.client.0.smithi027.stdout: "tunables": { 2020-09-12T14:00:47.161 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_local_tries": 0, 2020-09-12T14:00:47.161 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_local_fallback_tries": 0, 2020-09-12T14:00:47.162 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_total_tries": 50, 2020-09-12T14:00:47.162 INFO:tasks.workunit.client.0.smithi027.stdout: "chooseleaf_descend_once": 1, 2020-09-12T14:00:47.162 INFO:tasks.workunit.client.0.smithi027.stdout: "chooseleaf_vary_r": 1, 2020-09-12T14:00:47.162 INFO:tasks.workunit.client.0.smithi027.stdout: "chooseleaf_stable": 1, 2020-09-12T14:00:47.162 INFO:tasks.workunit.client.0.smithi027.stdout: "straw_calc_version": 1, 2020-09-12T14:00:47.162 INFO:tasks.workunit.client.0.smithi027.stdout: "allowed_bucket_algs": 54, 2020-09-12T14:00:47.162 INFO:tasks.workunit.client.0.smithi027.stdout: "profile": "jewel", 2020-09-12T14:00:47.162 INFO:tasks.workunit.client.0.smithi027.stdout: "optimal_tunables": 1, 2020-09-12T14:00:47.163 INFO:tasks.workunit.client.0.smithi027.stdout: "legacy_tunables": 0, 2020-09-12T14:00:47.163 INFO:tasks.workunit.client.0.smithi027.stdout: "minimum_required_version": "jewel", 2020-09-12T14:00:47.163 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables": 1, 2020-09-12T14:00:47.163 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables2": 1, 2020-09-12T14:00:47.163 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v2_rules": 0, 2020-09-12T14:00:47.163 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables3": 1, 2020-09-12T14:00:47.163 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v3_rules": 0, 2020-09-12T14:00:47.163 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v4_buckets": 1, 2020-09-12T14:00:47.164 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables5": 1, 2020-09-12T14:00:47.164 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v5_rules": 0 2020-09-12T14:00:47.164 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.164 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_args": {} 2020-09-12T14:00:47.164 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.164 INFO:tasks.workunit.client.0.smithi027.stdout: "fsmap": { 2020-09-12T14:00:47.164 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 1, 2020-09-12T14:00:47.164 INFO:tasks.workunit.client.0.smithi027.stdout: "default_fscid": -1, 2020-09-12T14:00:47.165 INFO:tasks.workunit.client.0.smithi027.stdout: "compat": { 2020-09-12T14:00:47.165 INFO:tasks.workunit.client.0.smithi027.stdout: "compat": {}, 2020-09-12T14:00:47.165 INFO:tasks.workunit.client.0.smithi027.stdout: "ro_compat": {}, 2020-09-12T14:00:47.165 INFO:tasks.workunit.client.0.smithi027.stdout: "incompat": { 2020-09-12T14:00:47.165 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_1": "base v0.20", 2020-09-12T14:00:47.165 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_2": "client writeable ranges", 2020-09-12T14:00:47.165 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_3": "default file layouts on dirs", 2020-09-12T14:00:47.166 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_4": "dir inode in separate object", 2020-09-12T14:00:47.166 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_5": "mds uses versioned encoding", 2020-09-12T14:00:47.166 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_6": "dirfrag is stored in omap", 2020-09-12T14:00:47.166 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_8": "no anchor table", 2020-09-12T14:00:47.166 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_9": "file layout v2", 2020-09-12T14:00:47.166 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_10": "snaprealm v2" 2020-09-12T14:00:47.166 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.166 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.167 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_flags": { 2020-09-12T14:00:47.167 INFO:tasks.workunit.client.0.smithi027.stdout: "enable_multiple": false, 2020-09-12T14:00:47.167 INFO:tasks.workunit.client.0.smithi027.stdout: "ever_enabled_multiple": false 2020-09-12T14:00:47.167 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.167 INFO:tasks.workunit.client.0.smithi027.stdout: "standbys": [], 2020-09-12T14:00:47.167 INFO:tasks.workunit.client.0.smithi027.stdout: "filesystems": [] 2020-09-12T14:00:47.167 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.167 INFO:tasks.workunit.client.0.smithi027.stdout: "mdsmap_first_committed": 1, 2020-09-12T14:00:47.168 INFO:tasks.workunit.client.0.smithi027.stdout: "mdsmap_last_committed": 1, 2020-09-12T14:00:47.168 INFO:tasks.workunit.client.0.smithi027.stdout: "auth": { 2020-09-12T14:00:47.168 INFO:tasks.workunit.client.0.smithi027.stdout: "first_committed": 1, 2020-09-12T14:00:47.168 INFO:tasks.workunit.client.0.smithi027.stdout: "last_committed": 5, 2020-09-12T14:00:47.168 INFO:tasks.workunit.client.0.smithi027.stdout: "num_secrets": 3 2020-09-12T14:00:47.168 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.168 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg": 1, 2020-09-12T14:00:47.168 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_active": 1, 2020-09-12T14:00:47.169 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_unknown": 0, 2020-09-12T14:00:47.169 INFO:tasks.workunit.client.0.smithi027.stdout: "num_osd": 3, 2020-09-12T14:00:47.169 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_sum": { 2020-09-12T14:00:47.169 INFO:tasks.workunit.client.0.smithi027.stdout: "stat_sum": { 2020-09-12T14:00:47.169 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes": 48300, 2020-09-12T14:00:47.169 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects": 100, 2020-09-12T14:00:47.169 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_clones": 0, 2020-09-12T14:00:47.169 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_copies": 300, 2020-09-12T14:00:47.170 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing_on_primary": 0, 2020-09-12T14:00:47.170 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing": 0, 2020-09-12T14:00:47.170 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_degraded": 100, 2020-09-12T14:00:47.170 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_misplaced": 0, 2020-09-12T14:00:47.170 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_unfound": 0, 2020-09-12T14:00:47.170 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_dirty": 100, 2020-09-12T14:00:47.170 INFO:tasks.workunit.client.0.smithi027.stdout: "num_whiteouts": 0, 2020-09-12T14:00:47.171 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read": 0, 2020-09-12T14:00:47.171 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read_kb": 0, 2020-09-12T14:00:47.171 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write": 100, 2020-09-12T14:00:47.171 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write_kb": 100, 2020-09-12T14:00:47.171 INFO:tasks.workunit.client.0.smithi027.stdout: "num_scrub_errors": 0, 2020-09-12T14:00:47.171 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shallow_scrub_errors": 0, 2020-09-12T14:00:47.171 INFO:tasks.workunit.client.0.smithi027.stdout: "num_deep_scrub_errors": 0, 2020-09-12T14:00:47.171 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_recovered": 0, 2020-09-12T14:00:47.172 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_recovered": 0, 2020-09-12T14:00:47.172 INFO:tasks.workunit.client.0.smithi027.stdout: "num_keys_recovered": 0, 2020-09-12T14:00:47.172 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_omap": 0, 2020-09-12T14:00:47.172 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_hit_set_archive": 0, 2020-09-12T14:00:47.172 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_hit_set_archive": 0, 2020-09-12T14:00:47.172 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush": 0, 2020-09-12T14:00:47.172 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_kb": 0, 2020-09-12T14:00:47.172 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict": 0, 2020-09-12T14:00:47.173 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_kb": 0, 2020-09-12T14:00:47.173 INFO:tasks.workunit.client.0.smithi027.stdout: "num_promote": 0, 2020-09-12T14:00:47.173 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_high": 0, 2020-09-12T14:00:47.173 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_low": 0, 2020-09-12T14:00:47.173 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_some": 0, 2020-09-12T14:00:47.173 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_full": 0, 2020-09-12T14:00:47.173 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_pinned": 0, 2020-09-12T14:00:47.173 INFO:tasks.workunit.client.0.smithi027.stdout: "num_legacy_snapsets": 0, 2020-09-12T14:00:47.174 INFO:tasks.workunit.client.0.smithi027.stdout: "num_large_omap_objects": 0, 2020-09-12T14:00:47.174 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_manifest": 0, 2020-09-12T14:00:47.174 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_bytes": 0, 2020-09-12T14:00:47.174 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_keys": 0, 2020-09-12T14:00:47.174 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_repaired": 0 2020-09-12T14:00:47.174 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.174 INFO:tasks.workunit.client.0.smithi027.stdout: "store_stats": { 2020-09-12T14:00:47.174 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 0, 2020-09-12T14:00:47.175 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 0, 2020-09-12T14:00:47.175 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:00:47.175 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 0, 2020-09-12T14:00:47.175 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 0, 2020-09-12T14:00:47.175 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:00:47.175 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:00:47.175 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:00:47.176 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 0, 2020-09-12T14:00:47.176 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 0 2020-09-12T14:00:47.176 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.176 INFO:tasks.workunit.client.0.smithi027.stdout: "log_size": 16, 2020-09-12T14:00:47.176 INFO:tasks.workunit.client.0.smithi027.stdout: "ondisk_log_size": 16, 2020-09-12T14:00:47.176 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 2, 2020-09-12T14:00:47.176 INFO:tasks.workunit.client.0.smithi027.stdout: "acting": 2, 2020-09-12T14:00:47.176 INFO:tasks.workunit.client.0.smithi027.stdout: "num_store_stats": 0 2020-09-12T14:00:47.177 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.177 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_sum": { 2020-09-12T14:00:47.177 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 0, 2020-09-12T14:00:47.177 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 0, 2020-09-12T14:00:47.177 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pgs": 3, 2020-09-12T14:00:47.177 INFO:tasks.workunit.client.0.smithi027.stdout: "num_osds": 3, 2020-09-12T14:00:47.177 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_osds": 3, 2020-09-12T14:00:47.177 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_omap_osds": 3, 2020-09-12T14:00:47.178 INFO:tasks.workunit.client.0.smithi027.stdout: "kb": 314572800, 2020-09-12T14:00:47.178 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used": 18168, 2020-09-12T14:00:47.178 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_data": 2072, 2020-09-12T14:00:47.178 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_omap": 1, 2020-09-12T14:00:47.178 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_meta": 16062, 2020-09-12T14:00:47.178 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_avail": 314554632, 2020-09-12T14:00:47.178 INFO:tasks.workunit.client.0.smithi027.stdout: "statfs": { 2020-09-12T14:00:47.178 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 322122547200, 2020-09-12T14:00:47.179 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 322103943168, 2020-09-12T14:00:47.179 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:00:47.179 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 2121728, 2020-09-12T14:00:47.179 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 403228, 2020-09-12T14:00:47.179 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:00:47.179 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:00:47.179 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:00:47.179 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 1928, 2020-09-12T14:00:47.180 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 16447608 2020-09-12T14:00:47.180 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.180 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_peers": [], 2020-09-12T14:00:47.180 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_trim_queue_len": 0, 2020-09-12T14:00:47.180 INFO:tasks.workunit.client.0.smithi027.stdout: "num_snap_trimming": 0, 2020-09-12T14:00:47.180 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shards_repaired": 0, 2020-09-12T14:00:47.180 INFO:tasks.workunit.client.0.smithi027.stdout: "op_queue_age_hist": { 2020-09-12T14:00:47.181 INFO:tasks.workunit.client.0.smithi027.stdout: "histogram": [], 2020-09-12T14:00:47.181 INFO:tasks.workunit.client.0.smithi027.stdout: "upper_bound": 1 2020-09-12T14:00:47.181 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.181 INFO:tasks.workunit.client.0.smithi027.stdout: "perf_stat": { 2020-09-12T14:00:47.181 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ms": 0, 2020-09-12T14:00:47.181 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ms": 0, 2020-09-12T14:00:47.181 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ns": 0, 2020-09-12T14:00:47.181 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ns": 0 2020-09-12T14:00:47.182 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.182 INFO:tasks.workunit.client.0.smithi027.stdout: "alerts": [], 2020-09-12T14:00:47.182 INFO:tasks.workunit.client.0.smithi027.stdout: "network_ping_times": [] 2020-09-12T14:00:47.182 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.182 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_sum_by_class": { 2020-09-12T14:00:47.182 INFO:tasks.workunit.client.0.smithi027.stdout: "hdd": { 2020-09-12T14:00:47.182 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 0, 2020-09-12T14:00:47.182 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 0, 2020-09-12T14:00:47.183 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pgs": 3, 2020-09-12T14:00:47.183 INFO:tasks.workunit.client.0.smithi027.stdout: "num_osds": 3, 2020-09-12T14:00:47.183 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_osds": 3, 2020-09-12T14:00:47.183 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_omap_osds": 3, 2020-09-12T14:00:47.183 INFO:tasks.workunit.client.0.smithi027.stdout: "kb": 314572800, 2020-09-12T14:00:47.183 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used": 18168, 2020-09-12T14:00:47.183 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_data": 2072, 2020-09-12T14:00:47.183 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_omap": 1, 2020-09-12T14:00:47.184 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_meta": 16062, 2020-09-12T14:00:47.184 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_avail": 314554632, 2020-09-12T14:00:47.184 INFO:tasks.workunit.client.0.smithi027.stdout: "statfs": { 2020-09-12T14:00:47.184 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 322122547200, 2020-09-12T14:00:47.184 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 322103943168, 2020-09-12T14:00:47.184 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:00:47.184 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 2121728, 2020-09-12T14:00:47.185 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 403228, 2020-09-12T14:00:47.185 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:00:47.185 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:00:47.185 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:00:47.185 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 1928, 2020-09-12T14:00:47.185 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 16447608 2020-09-12T14:00:47.185 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.185 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_peers": [], 2020-09-12T14:00:47.186 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_trim_queue_len": 0, 2020-09-12T14:00:47.186 INFO:tasks.workunit.client.0.smithi027.stdout: "num_snap_trimming": 0, 2020-09-12T14:00:47.186 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shards_repaired": 0, 2020-09-12T14:00:47.186 INFO:tasks.workunit.client.0.smithi027.stdout: "op_queue_age_hist": { 2020-09-12T14:00:47.186 INFO:tasks.workunit.client.0.smithi027.stdout: "histogram": [], 2020-09-12T14:00:47.186 INFO:tasks.workunit.client.0.smithi027.stdout: "upper_bound": 1 2020-09-12T14:00:47.186 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.186 INFO:tasks.workunit.client.0.smithi027.stdout: "perf_stat": { 2020-09-12T14:00:47.187 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ms": 0, 2020-09-12T14:00:47.187 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ms": 0, 2020-09-12T14:00:47.187 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ns": 0, 2020-09-12T14:00:47.187 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ns": 0 2020-09-12T14:00:47.187 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.187 INFO:tasks.workunit.client.0.smithi027.stdout: "alerts": [], 2020-09-12T14:00:47.187 INFO:tasks.workunit.client.0.smithi027.stdout: "network_ping_times": [] 2020-09-12T14:00:47.187 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.188 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.188 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_stats": [ 2020-09-12T14:00:47.188 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.188 INFO:tasks.workunit.client.0.smithi027.stdout: "poolid": 1, 2020-09-12T14:00:47.188 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg": 1, 2020-09-12T14:00:47.188 INFO:tasks.workunit.client.0.smithi027.stdout: "stat_sum": { 2020-09-12T14:00:47.188 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes": 48300, 2020-09-12T14:00:47.188 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects": 100, 2020-09-12T14:00:47.189 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_clones": 0, 2020-09-12T14:00:47.189 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_copies": 300, 2020-09-12T14:00:47.189 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing_on_primary": 0, 2020-09-12T14:00:47.189 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing": 0, 2020-09-12T14:00:47.189 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_degraded": 100, 2020-09-12T14:00:47.189 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_misplaced": 0, 2020-09-12T14:00:47.189 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_unfound": 0, 2020-09-12T14:00:47.189 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_dirty": 100, 2020-09-12T14:00:47.190 INFO:tasks.workunit.client.0.smithi027.stdout: "num_whiteouts": 0, 2020-09-12T14:00:47.190 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read": 0, 2020-09-12T14:00:47.190 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read_kb": 0, 2020-09-12T14:00:47.190 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write": 100, 2020-09-12T14:00:47.190 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write_kb": 100, 2020-09-12T14:00:47.190 INFO:tasks.workunit.client.0.smithi027.stdout: "num_scrub_errors": 0, 2020-09-12T14:00:47.190 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shallow_scrub_errors": 0, 2020-09-12T14:00:47.191 INFO:tasks.workunit.client.0.smithi027.stdout: "num_deep_scrub_errors": 0, 2020-09-12T14:00:47.191 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_recovered": 0, 2020-09-12T14:00:47.191 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_recovered": 0, 2020-09-12T14:00:47.191 INFO:tasks.workunit.client.0.smithi027.stdout: "num_keys_recovered": 0, 2020-09-12T14:00:47.191 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_omap": 0, 2020-09-12T14:00:47.191 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_hit_set_archive": 0, 2020-09-12T14:00:47.191 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_hit_set_archive": 0, 2020-09-12T14:00:47.191 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush": 0, 2020-09-12T14:00:47.192 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_kb": 0, 2020-09-12T14:00:47.192 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict": 0, 2020-09-12T14:00:47.192 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_kb": 0, 2020-09-12T14:00:47.192 INFO:tasks.workunit.client.0.smithi027.stdout: "num_promote": 0, 2020-09-12T14:00:47.192 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_high": 0, 2020-09-12T14:00:47.192 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_low": 0, 2020-09-12T14:00:47.192 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_some": 0, 2020-09-12T14:00:47.192 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_full": 0, 2020-09-12T14:00:47.193 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_pinned": 0, 2020-09-12T14:00:47.193 INFO:tasks.workunit.client.0.smithi027.stdout: "num_legacy_snapsets": 0, 2020-09-12T14:00:47.193 INFO:tasks.workunit.client.0.smithi027.stdout: "num_large_omap_objects": 0, 2020-09-12T14:00:47.193 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_manifest": 0, 2020-09-12T14:00:47.193 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_bytes": 0, 2020-09-12T14:00:47.193 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_keys": 0, 2020-09-12T14:00:47.193 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_repaired": 0 2020-09-12T14:00:47.193 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.194 INFO:tasks.workunit.client.0.smithi027.stdout: "store_stats": { 2020-09-12T14:00:47.194 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 0, 2020-09-12T14:00:47.194 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 0, 2020-09-12T14:00:47.194 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:00:47.194 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 1220608, 2020-09-12T14:00:47.194 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 143228, 2020-09-12T14:00:47.194 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:00:47.194 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:00:47.195 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:00:47.195 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 0, 2020-09-12T14:00:47.195 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 0 2020-09-12T14:00:47.195 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.195 INFO:tasks.workunit.client.0.smithi027.stdout: "log_size": 16, 2020-09-12T14:00:47.195 INFO:tasks.workunit.client.0.smithi027.stdout: "ondisk_log_size": 16, 2020-09-12T14:00:47.195 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 2, 2020-09-12T14:00:47.195 INFO:tasks.workunit.client.0.smithi027.stdout: "acting": 2, 2020-09-12T14:00:47.196 INFO:tasks.workunit.client.0.smithi027.stdout: "num_store_stats": 3 2020-09-12T14:00:47.196 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.196 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:00:47.196 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_stats": [ 2020-09-12T14:00:47.196 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.196 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:00:47.196 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 150323855371 2020-09-12T14:00:47.197 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.197 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.197 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:00:47.197 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 42949672977 2020-09-12T14:00:47.197 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.197 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.197 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:00:47.197 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 154618822666 2020-09-12T14:00:47.198 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.198 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:00:47.198 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_by_state": [ 2020-09-12T14:00:47.198 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.198 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "active+undersized+degraded", 2020-09-12T14:00:47.198 INFO:tasks.workunit.client.0.smithi027.stdout: "num": 1 2020-09-12T14:00:47.198 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.198 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:00:47.199 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_by_osd": [ 2020-09-12T14:00:47.199 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.199 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:00:47.199 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 0, 2020-09-12T14:00:47.199 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 1, 2020-09-12T14:00:47.199 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T14:00:47.199 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.199 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.200 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:00:47.200 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 0, 2020-09-12T14:00:47.200 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 0, 2020-09-12T14:00:47.200 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T14:00:47.200 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.200 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.200 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:00:47.200 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 1, 2020-09-12T14:00:47.201 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 1, 2020-09-12T14:00:47.201 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T14:00:47.201 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.201 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:00:47.201 INFO:tasks.workunit.client.0.smithi027.stdout: "purged_snaps": [ 2020-09-12T14:00:47.201 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:00:47.201 INFO:tasks.workunit.client.0.smithi027.stdout: "pool": 1, 2020-09-12T14:00:47.202 INFO:tasks.workunit.client.0.smithi027.stdout: "purged_snaps": {} 2020-09-12T14:00:47.202 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.202 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:00:47.202 INFO:tasks.workunit.client.0.smithi027.stdout: "servicemap": { 2020-09-12T14:00:47.202 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 4, 2020-09-12T14:00:47.202 INFO:tasks.workunit.client.0.smithi027.stdout: "modified": "2020-09-12T14:00:10.726846+0000", 2020-09-12T14:00:47.202 INFO:tasks.workunit.client.0.smithi027.stdout: "services": { 2020-09-12T14:00:47.202 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": { 2020-09-12T14:00:47.203 INFO:tasks.workunit.client.0.smithi027.stdout: "daemons": { 2020-09-12T14:00:47.203 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": "", 2020-09-12T14:00:47.203 INFO:tasks.workunit.client.0.smithi027.stdout: "0": { 2020-09-12T14:00:47.203 INFO:tasks.workunit.client.0.smithi027.stdout: "start_epoch": 0, 2020-09-12T14:00:47.203 INFO:tasks.workunit.client.0.smithi027.stdout: "start_stamp": "0.000000", 2020-09-12T14:00:47.203 INFO:tasks.workunit.client.0.smithi027.stdout: "gid": 0, 2020-09-12T14:00:47.203 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "(unrecognized address family 0)/0", 2020-09-12T14:00:47.203 INFO:tasks.workunit.client.0.smithi027.stdout: "metadata": {}, 2020-09-12T14:00:47.204 INFO:tasks.workunit.client.0.smithi027.stdout: "task_status": {} 2020-09-12T14:00:47.204 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.204 INFO:tasks.workunit.client.0.smithi027.stdout: "2": { 2020-09-12T14:00:47.204 INFO:tasks.workunit.client.0.smithi027.stdout: "start_epoch": 0, 2020-09-12T14:00:47.204 INFO:tasks.workunit.client.0.smithi027.stdout: "start_stamp": "0.000000", 2020-09-12T14:00:47.204 INFO:tasks.workunit.client.0.smithi027.stdout: "gid": 0, 2020-09-12T14:00:47.204 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "(unrecognized address family 0)/0", 2020-09-12T14:00:47.204 INFO:tasks.workunit.client.0.smithi027.stdout: "metadata": {}, 2020-09-12T14:00:47.205 INFO:tasks.workunit.client.0.smithi027.stdout: "task_status": {} 2020-09-12T14:00:47.205 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.205 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.205 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.205 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.205 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:00:47.205 INFO:tasks.workunit.client.0.smithi027.stdout: "paxos": { 2020-09-12T14:00:47.205 INFO:tasks.workunit.client.0.smithi027.stdout: "first_committed": 1, 2020-09-12T14:00:47.206 INFO:tasks.workunit.client.0.smithi027.stdout: "last_committed": 353, 2020-09-12T14:00:47.206 INFO:tasks.workunit.client.0.smithi027.stdout: "last_pn": 0, 2020-09-12T14:00:47.206 INFO:tasks.workunit.client.0.smithi027.stdout: "accepted_pn": 0 2020-09-12T14:00:47.206 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:00:47.206 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:00:47.207 INFO:tasks.workunit.client.0.smithi027.stderr:report 3287716834 2020-09-12T14:00:47.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1604: wait_for_clean: return 1 2020-09-12T14:00:47.207 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:528: TEST_divergent_2: expr 2 + 2 2020-09-12T14:00:47.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:528: TEST_divergent_2: objname=existing_4 2020-09-12T14:00:47.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:529: TEST_divergent_2: echo 'writing non-divergent object existing_4' 2020-09-12T14:00:47.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:530: TEST_divergent_2: ceph pg dump pgs 2020-09-12T14:00:47.208 INFO:tasks.workunit.client.0.smithi027.stdout:writing non-divergent object existing_4 2020-09-12T14:00:47.446 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:00:47.447 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 0 100 0 0 48300 0 0 16 16 active+undersized+degraded 2020-09-12T14:00:10.331732+0000 29'100 39:120 [0,2] 0 [0,2] 0 0'0 2020-09-12T13:59:19.138081+0000 0'0 2020-09-12T13:59:19.138081+0000 0 2020-09-12T14:00:47.447 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:00:47.447 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:00:47.448 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:00:47.457 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:531: TEST_divergent_2: rados -p test put existing_4 /etc/resolv.conf 2020-09-12T14:00:47.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:533: TEST_divergent_2: WAIT_FOR_CLEAN_TIMEOUT=20 2020-09-12T14:00:47.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:533: TEST_divergent_2: wait_for_clean 2020-09-12T14:00:47.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:00:47.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:00:47.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:00:47.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:00:47.581 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 20 .1 2020-09-12T14:00:47.582 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:00:47.582 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:00:47.582 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:00:47.583 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:00:47.583 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:00:47.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:00:47.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:00:47.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:00:47.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:00:47.641 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:00:48.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:00:48.066 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:00:48.066 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T14:00:48.066 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:00:48.066 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:00:48.066 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:00:48.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=150323855373 2020-09-12T14:00:48.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 150323855373 2020-09-12T14:00:48.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-150323855373' 2020-09-12T14:00:48.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:00:48.213 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:00:48.320 INFO:tasks.workunit.client.0.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-09-12T14:00:48.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq= 2020-09-12T14:00:48.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z '' 2020-09-12T14:00:48.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2075: flush_pg_stats: continue 2020-09-12T14:00:48.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:00:48.322 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:00:48.468 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=154618822669 2020-09-12T14:00:48.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 154618822669 2020-09-12T14:00:48.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-150323855373 2-154618822669' 2020-09-12T14:00:48.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:00:48.469 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-150323855373 2020-09-12T14:00:48.470 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:00:48.470 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:00:48.470 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-150323855373 2020-09-12T14:00:48.470 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:00:48.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=150323855373 2020-09-12T14:00:48.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 150323855373' 2020-09-12T14:00:48.471 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 150323855373 2020-09-12T14:00:48.472 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:00:48.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 150323855373 -lt 150323855373 2020-09-12T14:00:48.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:00:48.901 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-154618822669 2020-09-12T14:00:48.901 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:00:48.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:00:48.902 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-154618822669 2020-09-12T14:00:48.902 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:00:48.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=154618822669 2020-09-12T14:00:48.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 154618822669' 2020-09-12T14:00:48.903 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 154618822669 2020-09-12T14:00:48.904 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:00:49.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 154618822669 -lt 154618822669 2020-09-12T14:00:49.314 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:00:49.314 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:00:49.314 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:00:49.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 1 == 0 2020-09-12T14:00:49.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:00:49.847 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:00:49.847 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:00:49.847 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:00:49.848 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:00:49.848 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:00:49.848 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:00:50.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:00:50.245 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:00:50.245 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:00:50.245 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:00:50.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:00:50.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' -1 2020-09-12T14:00:50.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1598: wait_for_clean: loop=0 2020-09-12T14:00:50.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1599: wait_for_clean: num_active_clean=0 2020-09-12T14:00:50.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:00:50.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:00:50.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:00:50.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:00:50.889 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:00:50.889 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:00:50.889 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:00:50.889 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:00:50.889 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:00:50.890 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:00:51.291 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:00:51.292 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:00:51.293 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:00:51.293 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:00:51.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:00:51.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:00:51.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:00:51.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:00:51.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:00:51.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:00:51.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:00:51.829 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:00:51.829 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:00:52.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:00:52.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:00:52.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 1 >= 8 )) 2020-09-12T14:00:52.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:00:52.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.2 2020-09-12T14:00:52.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:00:52.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:00:52.569 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:00:52.569 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:00:52.569 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:00:52.569 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:00:52.569 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:00:52.569 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:00:52.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:00:52.969 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:00:52.969 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:00:52.969 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:00:53.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:00:53.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:00:53.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:00:53.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:00:53.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:00:53.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:00:53.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:00:53.507 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:00:53.507 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:00:54.038 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:00:54.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:00:54.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 2 >= 8 )) 2020-09-12T14:00:54.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:00:54.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.4 2020-09-12T14:00:54.439 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:00:54.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:00:54.440 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:00:54.440 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:00:54.440 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:00:54.440 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:00:54.441 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:00:54.441 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:00:54.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:00:54.840 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:00:54.841 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:00:54.841 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:00:55.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:00:55.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:00:55.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:00:55.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:00:55.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:00:55.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:00:55.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:00:55.375 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:00:55.376 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:00:55.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:00:55.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:00:55.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 3 >= 8 )) 2020-09-12T14:00:55.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:00:55.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.8 2020-09-12T14:00:56.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:00:56.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:00:56.712 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:00:56.713 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:00:56.713 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:00:56.713 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:00:56.713 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:00:56.713 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:00:57.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:00:57.125 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:00:57.126 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:00:57.126 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:00:57.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:00:57.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:00:57.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:00:57.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:00:57.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:00:57.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:00:57.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:00:57.658 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:00:57.658 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:00:58.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:00:58.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:00:58.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 4 >= 8 )) 2020-09-12T14:00:58.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:00:58.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 1.6 2020-09-12T14:00:59.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:00:59.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:00:59.820 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:00:59.821 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:00:59.821 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:00:59.821 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:00:59.821 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:00:59.821 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:01:00.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:01:00.238 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:01:00.238 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:01:00.238 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:01:00.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:01:00.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:01:00.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:01:00.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:01:00.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:01:00.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:01:00.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:01:00.776 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:01:00.776 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:01:01.315 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:01:01.316 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:01:01.316 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 5 >= 8 )) 2020-09-12T14:01:01.316 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:01:01.316 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 3.2 2020-09-12T14:01:04.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:01:04.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:01:04.517 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:01:04.518 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:01:04.518 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:01:04.518 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:01:04.518 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:01:04.518 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:01:04.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:01:04.926 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:01:04.927 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:01:04.927 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:01:05.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:01:05.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:01:05.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:01:05.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:01:05.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:01:05.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:01:05.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:01:05.461 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:01:05.462 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:01:05.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:01:05.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:01:05.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 6 >= 8 )) 2020-09-12T14:01:05.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:01:05.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 6.4 2020-09-12T14:01:12.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:01:12.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:01:12.393 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:01:12.393 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:01:12.393 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:01:12.394 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:01:12.394 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:01:12.394 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:01:12.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:01:12.808 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:01:12.808 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:01:12.808 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:01:13.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:01:13.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:01:13.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:01:13.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:01:13.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:01:13.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:01:13.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:01:13.349 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:01:13.349 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:01:13.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:01:13.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:01:13.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 7 >= 8 )) 2020-09-12T14:01:13.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:01:13.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 7.3 2020-09-12T14:01:21.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:01:21.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:01:21.187 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:01:21.187 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:01:21.187 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:01:21.187 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:01:21.187 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:01:21.187 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:01:21.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:01:21.592 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:01:21.592 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:01:21.592 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:01:22.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:01:22.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:01:22.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:01:22.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:01:22.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:01:22.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:01:22.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:01:22.121 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:01:22.122 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:01:22.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:01:22.653 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:01:22.653 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 8 >= 8 )) 2020-09-12T14:01:22.653 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: wait_for_clean: ceph report 2020-09-12T14:01:23.202 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:01:23.203 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_fingerprint": "7db1c797-8407-4b23-93e4-602672e04493", 2020-09-12T14:01:23.203 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:01:23.203 INFO:tasks.workunit.client.0.smithi027.stdout: "commit": "f346cc73d15a568d3bec1c99a58953de76f57b85", 2020-09-12T14:01:23.203 INFO:tasks.workunit.client.0.smithi027.stdout: "timestamp": "2020-09-12T14:01:23.201199+0000", 2020-09-12T14:01:23.203 INFO:tasks.workunit.client.0.smithi027.stdout: "tag": "", 2020-09-12T14:01:23.203 INFO:tasks.workunit.client.0.smithi027.stdout: "health": { 2020-09-12T14:01:23.204 INFO:tasks.workunit.client.0.smithi027.stdout: "status": "HEALTH_WARN", 2020-09-12T14:01:23.204 INFO:tasks.workunit.client.0.smithi027.stdout: "checks": { 2020-09-12T14:01:23.204 INFO:tasks.workunit.client.0.smithi027.stdout: "OSDMAP_FLAGS": { 2020-09-12T14:01:23.204 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:01:23.204 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:01:23.204 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "nodown,noout,noin flag(s) set", 2020-09-12T14:01:23.204 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 17 2020-09-12T14:01:23.205 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.205 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [], 2020-09-12T14:01:23.205 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:01:23.205 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.205 INFO:tasks.workunit.client.0.smithi027.stdout: "OSD_DOWN": { 2020-09-12T14:01:23.205 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:01:23.205 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:01:23.205 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "1 osds down", 2020-09-12T14:01:23.206 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 1 2020-09-12T14:01:23.206 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.206 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [ 2020-09-12T14:01:23.206 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.206 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "osd.1 (root=default,host=smithi027) is down" 2020-09-12T14:01:23.206 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.206 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:01:23.207 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:01:23.207 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.207 INFO:tasks.workunit.client.0.smithi027.stdout: "PG_DEGRADED": { 2020-09-12T14:01:23.207 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:01:23.207 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:01:23.207 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 1 pg degraded, 1 pg undersized", 2020-09-12T14:01:23.207 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 2 2020-09-12T14:01:23.208 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.208 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [ 2020-09-12T14:01:23.208 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.208 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.0 is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:01:23.208 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.208 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:01:23.208 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:01:23.208 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.209 INFO:tasks.workunit.client.0.smithi027.stdout: "POOL_APP_NOT_ENABLED": { 2020-09-12T14:01:23.209 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:01:23.209 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:01:23.209 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "1 pool(s) do not have an application enabled", 2020-09-12T14:01:23.209 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 1 2020-09-12T14:01:23.209 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.209 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [ 2020-09-12T14:01:23.210 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.210 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "application not enabled on pool 'test'" 2020-09-12T14:01:23.210 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.210 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.210 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2020-09-12T14:01:23.210 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.210 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:01:23.211 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:01:23.211 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.211 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.211 INFO:tasks.workunit.client.0.smithi027.stdout: "mutes": [] 2020-09-12T14:01:23.211 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.211 INFO:tasks.workunit.client.0.smithi027.stdout: "monmap_first_committed": 1, 2020-09-12T14:01:23.211 INFO:tasks.workunit.client.0.smithi027.stdout: "monmap_last_committed": 1, 2020-09-12T14:01:23.212 INFO:tasks.workunit.client.0.smithi027.stdout: "monmap": { 2020-09-12T14:01:23.212 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 1, 2020-09-12T14:01:23.212 INFO:tasks.workunit.client.0.smithi027.stdout: "fsid": "7e90bad3-7973-4a8c-be65-6326ab931fe8", 2020-09-12T14:01:23.212 INFO:tasks.workunit.client.0.smithi027.stdout: "modified": "2020-09-12T13:58:51.776233Z", 2020-09-12T14:01:23.212 INFO:tasks.workunit.client.0.smithi027.stdout: "created": "2020-09-12T13:58:51.776233Z", 2020-09-12T14:01:23.212 INFO:tasks.workunit.client.0.smithi027.stdout: "min_mon_release": 16, 2020-09-12T14:01:23.212 INFO:tasks.workunit.client.0.smithi027.stdout: "min_mon_release_name": "pacific", 2020-09-12T14:01:23.213 INFO:tasks.workunit.client.0.smithi027.stdout: "features": { 2020-09-12T14:01:23.213 INFO:tasks.workunit.client.0.smithi027.stdout: "persistent": [ 2020-09-12T14:01:23.213 INFO:tasks.workunit.client.0.smithi027.stdout: "kraken", 2020-09-12T14:01:23.213 INFO:tasks.workunit.client.0.smithi027.stdout: "luminous", 2020-09-12T14:01:23.213 INFO:tasks.workunit.client.0.smithi027.stdout: "mimic", 2020-09-12T14:01:23.213 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap-prune", 2020-09-12T14:01:23.214 INFO:tasks.workunit.client.0.smithi027.stdout: "nautilus", 2020-09-12T14:01:23.214 INFO:tasks.workunit.client.0.smithi027.stdout: "octopus", 2020-09-12T14:01:23.214 INFO:tasks.workunit.client.0.smithi027.stdout: "pacific" 2020-09-12T14:01:23.214 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:01:23.214 INFO:tasks.workunit.client.0.smithi027.stdout: "optional": [] 2020-09-12T14:01:23.214 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.215 INFO:tasks.workunit.client.0.smithi027.stdout: "mons": [ 2020-09-12T14:01:23.215 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.215 INFO:tasks.workunit.client.0.smithi027.stdout: "rank": 0, 2020-09-12T14:01:23.215 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "a", 2020-09-12T14:01:23.215 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:01:23.215 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:01:23.215 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.215 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:01:23.216 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:7115", 2020-09-12T14:01:23.216 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 0 2020-09-12T14:01:23.216 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.216 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.216 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.216 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:7115/0", 2020-09-12T14:01:23.216 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:7115/0", 2020-09-12T14:01:23.216 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 0, 2020-09-12T14:01:23.217 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 0 2020-09-12T14:01:23.217 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.217 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.217 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.217 INFO:tasks.workunit.client.0.smithi027.stdout: "quorum": [ 2020-09-12T14:01:23.217 INFO:tasks.workunit.client.0.smithi027.stdout: 0 2020-09-12T14:01:23.217 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:01:23.218 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap": { 2020-09-12T14:01:23.218 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 39, 2020-09-12T14:01:23.218 INFO:tasks.workunit.client.0.smithi027.stdout: "fsid": "7e90bad3-7973-4a8c-be65-6326ab931fe8", 2020-09-12T14:01:23.218 INFO:tasks.workunit.client.0.smithi027.stdout: "created": "2020-09-12T13:58:52.534797+0000", 2020-09-12T14:01:23.218 INFO:tasks.workunit.client.0.smithi027.stdout: "modified": "2020-09-12T14:00:09.995790+0000", 2020-09-12T14:01:23.218 INFO:tasks.workunit.client.0.smithi027.stdout: "last_up_change": "2020-09-12T14:00:08.455706+0000", 2020-09-12T14:01:23.218 INFO:tasks.workunit.client.0.smithi027.stdout: "last_in_change": "2020-09-12T14:00:09.716648+0000", 2020-09-12T14:01:23.219 INFO:tasks.workunit.client.0.smithi027.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-09-12T14:01:23.219 INFO:tasks.workunit.client.0.smithi027.stdout: "flags_num": 5800384, 2020-09-12T14:01:23.219 INFO:tasks.workunit.client.0.smithi027.stdout: "flags_set": [ 2020-09-12T14:01:23.219 INFO:tasks.workunit.client.0.smithi027.stdout: "nodown", 2020-09-12T14:01:23.219 INFO:tasks.workunit.client.0.smithi027.stdout: "noin", 2020-09-12T14:01:23.219 INFO:tasks.workunit.client.0.smithi027.stdout: "noout", 2020-09-12T14:01:23.219 INFO:tasks.workunit.client.0.smithi027.stdout: "pglog_hardlimit", 2020-09-12T14:01:23.219 INFO:tasks.workunit.client.0.smithi027.stdout: "purged_snapdirs", 2020-09-12T14:01:23.220 INFO:tasks.workunit.client.0.smithi027.stdout: "recovery_deletes", 2020-09-12T14:01:23.220 INFO:tasks.workunit.client.0.smithi027.stdout: "sortbitwise" 2020-09-12T14:01:23.220 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:01:23.220 INFO:tasks.workunit.client.0.smithi027.stdout: "crush_version": 7, 2020-09-12T14:01:23.220 INFO:tasks.workunit.client.0.smithi027.stdout: "full_ratio": 0.99000000953674316, 2020-09-12T14:01:23.220 INFO:tasks.workunit.client.0.smithi027.stdout: "backfillfull_ratio": 0.99000000953674316, 2020-09-12T14:01:23.221 INFO:tasks.workunit.client.0.smithi027.stdout: "nearfull_ratio": 0.85000002384185791, 2020-09-12T14:01:23.221 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_snapshot": "", 2020-09-12T14:01:23.221 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_max": 1, 2020-09-12T14:01:23.221 INFO:tasks.workunit.client.0.smithi027.stdout: "max_osd": 3, 2020-09-12T14:01:23.221 INFO:tasks.workunit.client.0.smithi027.stdout: "require_min_compat_client": "luminous", 2020-09-12T14:01:23.221 INFO:tasks.workunit.client.0.smithi027.stdout: "min_compat_client": "jewel", 2020-09-12T14:01:23.221 INFO:tasks.workunit.client.0.smithi027.stdout: "require_osd_release": "pacific", 2020-09-12T14:01:23.221 INFO:tasks.workunit.client.0.smithi027.stdout: "pools": [ 2020-09-12T14:01:23.222 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.222 INFO:tasks.workunit.client.0.smithi027.stdout: "pool": 1, 2020-09-12T14:01:23.222 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_name": "test", 2020-09-12T14:01:23.222 INFO:tasks.workunit.client.0.smithi027.stdout: "create_time": "2020-09-12T13:59:19.085741+0000", 2020-09-12T14:01:23.222 INFO:tasks.workunit.client.0.smithi027.stdout: "flags": 1, 2020-09-12T14:01:23.222 INFO:tasks.workunit.client.0.smithi027.stdout: "flags_names": "hashpspool", 2020-09-12T14:01:23.223 INFO:tasks.workunit.client.0.smithi027.stdout: "type": 1, 2020-09-12T14:01:23.223 INFO:tasks.workunit.client.0.smithi027.stdout: "size": 3, 2020-09-12T14:01:23.223 INFO:tasks.workunit.client.0.smithi027.stdout: "min_size": 2, 2020-09-12T14:01:23.223 INFO:tasks.workunit.client.0.smithi027.stdout: "crush_rule": 0, 2020-09-12T14:01:23.223 INFO:tasks.workunit.client.0.smithi027.stdout: "object_hash": 2, 2020-09-12T14:01:23.223 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_autoscale_mode": "off", 2020-09-12T14:01:23.223 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_num": 1, 2020-09-12T14:01:23.223 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_placement_num": 1, 2020-09-12T14:01:23.224 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_placement_num_target": 1, 2020-09-12T14:01:23.224 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_num_target": 1, 2020-09-12T14:01:23.224 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_num_pending": 1, 2020-09-12T14:01:23.224 INFO:tasks.workunit.client.0.smithi027.stdout: "last_pg_merge_meta": { 2020-09-12T14:01:23.224 INFO:tasks.workunit.client.0.smithi027.stdout: "source_pgid": "0.0", 2020-09-12T14:01:23.224 INFO:tasks.workunit.client.0.smithi027.stdout: "ready_epoch": 0, 2020-09-12T14:01:23.225 INFO:tasks.workunit.client.0.smithi027.stdout: "last_epoch_started": 0, 2020-09-12T14:01:23.225 INFO:tasks.workunit.client.0.smithi027.stdout: "last_epoch_clean": 0, 2020-09-12T14:01:23.225 INFO:tasks.workunit.client.0.smithi027.stdout: "source_version": "0'0", 2020-09-12T14:01:23.225 INFO:tasks.workunit.client.0.smithi027.stdout: "target_version": "0'0" 2020-09-12T14:01:23.225 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.225 INFO:tasks.workunit.client.0.smithi027.stdout: "last_change": "29", 2020-09-12T14:01:23.225 INFO:tasks.workunit.client.0.smithi027.stdout: "last_force_op_resend": "0", 2020-09-12T14:01:23.226 INFO:tasks.workunit.client.0.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2020-09-12T14:01:23.226 INFO:tasks.workunit.client.0.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2020-09-12T14:01:23.226 INFO:tasks.workunit.client.0.smithi027.stdout: "auid": 0, 2020-09-12T14:01:23.226 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_mode": "selfmanaged", 2020-09-12T14:01:23.226 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_seq": 0, 2020-09-12T14:01:23.226 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_epoch": 0, 2020-09-12T14:01:23.226 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_snaps": [], 2020-09-12T14:01:23.227 INFO:tasks.workunit.client.0.smithi027.stdout: "removed_snaps": "[]", 2020-09-12T14:01:23.227 INFO:tasks.workunit.client.0.smithi027.stdout: "quota_max_bytes": 0, 2020-09-12T14:01:23.227 INFO:tasks.workunit.client.0.smithi027.stdout: "quota_max_objects": 0, 2020-09-12T14:01:23.227 INFO:tasks.workunit.client.0.smithi027.stdout: "tiers": [], 2020-09-12T14:01:23.227 INFO:tasks.workunit.client.0.smithi027.stdout: "tier_of": -1, 2020-09-12T14:01:23.227 INFO:tasks.workunit.client.0.smithi027.stdout: "read_tier": -1, 2020-09-12T14:01:23.227 INFO:tasks.workunit.client.0.smithi027.stdout: "write_tier": -1, 2020-09-12T14:01:23.227 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_mode": "none", 2020-09-12T14:01:23.228 INFO:tasks.workunit.client.0.smithi027.stdout: "target_max_bytes": 0, 2020-09-12T14:01:23.228 INFO:tasks.workunit.client.0.smithi027.stdout: "target_max_objects": 0, 2020-09-12T14:01:23.228 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-09-12T14:01:23.228 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-09-12T14:01:23.228 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2020-09-12T14:01:23.228 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_min_flush_age": 0, 2020-09-12T14:01:23.228 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_min_evict_age": 0, 2020-09-12T14:01:23.229 INFO:tasks.workunit.client.0.smithi027.stdout: "erasure_code_profile": "", 2020-09-12T14:01:23.229 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_params": { 2020-09-12T14:01:23.229 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "none" 2020-09-12T14:01:23.229 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.229 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_period": 0, 2020-09-12T14:01:23.229 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_count": 0, 2020-09-12T14:01:23.229 INFO:tasks.workunit.client.0.smithi027.stdout: "use_gmt_hitset": true, 2020-09-12T14:01:23.229 INFO:tasks.workunit.client.0.smithi027.stdout: "min_read_recency_for_promote": 0, 2020-09-12T14:01:23.230 INFO:tasks.workunit.client.0.smithi027.stdout: "min_write_recency_for_promote": 0, 2020-09-12T14:01:23.230 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2020-09-12T14:01:23.230 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_search_last_n": 0, 2020-09-12T14:01:23.230 INFO:tasks.workunit.client.0.smithi027.stdout: "grade_table": [], 2020-09-12T14:01:23.230 INFO:tasks.workunit.client.0.smithi027.stdout: "stripe_width": 0, 2020-09-12T14:01:23.230 INFO:tasks.workunit.client.0.smithi027.stdout: "expected_num_objects": 0, 2020-09-12T14:01:23.230 INFO:tasks.workunit.client.0.smithi027.stdout: "fast_read": false, 2020-09-12T14:01:23.231 INFO:tasks.workunit.client.0.smithi027.stdout: "options": {}, 2020-09-12T14:01:23.231 INFO:tasks.workunit.client.0.smithi027.stdout: "application_metadata": {} 2020-09-12T14:01:23.231 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.231 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:01:23.231 INFO:tasks.workunit.client.0.smithi027.stdout: "osds": [ 2020-09-12T14:01:23.231 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.231 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:01:23.232 INFO:tasks.workunit.client.0.smithi027.stdout: "uuid": "6e938613-9665-406c-b148-ce5bfd24c111", 2020-09-12T14:01:23.232 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 1, 2020-09-12T14:01:23.232 INFO:tasks.workunit.client.0.smithi027.stdout: "in": 1, 2020-09-12T14:01:23.232 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 1, 2020-09-12T14:01:23.232 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_affinity": 1, 2020-09-12T14:01:23.232 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_begin": 5, 2020-09-12T14:01:23.232 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_end": 29, 2020-09-12T14:01:23.233 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 35, 2020-09-12T14:01:23.233 INFO:tasks.workunit.client.0.smithi027.stdout: "up_thru": 38, 2020-09-12T14:01:23.233 INFO:tasks.workunit.client.0.smithi027.stdout: "down_at": 30, 2020-09-12T14:01:23.233 INFO:tasks.workunit.client.0.smithi027.stdout: "lost_at": 0, 2020-09-12T14:01:23.233 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:01:23.233 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:01:23.233 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.234 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:01:23.234 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6802", 2020-09-12T14:01:23.234 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37498 2020-09-12T14:01:23.234 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.234 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.234 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:01:23.234 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6803", 2020-09-12T14:01:23.235 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37498 2020-09-12T14:01:23.235 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.235 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.235 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.235 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addrs": { 2020-09-12T14:01:23.235 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:01:23.235 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.235 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:01:23.236 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6804", 2020-09-12T14:01:23.236 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37498 2020-09-12T14:01:23.236 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.236 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.236 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:01:23.236 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6805", 2020-09-12T14:01:23.236 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37498 2020-09-12T14:01:23.237 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.237 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.237 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.237 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addrs": { 2020-09-12T14:01:23.237 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:01:23.237 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.238 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:01:23.238 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6808", 2020-09-12T14:01:23.238 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37498 2020-09-12T14:01:23.238 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.238 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.238 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:01:23.238 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6809", 2020-09-12T14:01:23.239 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37498 2020-09-12T14:01:23.239 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.239 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.239 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.239 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addrs": { 2020-09-12T14:01:23.239 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:01:23.239 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.239 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:01:23.240 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6806", 2020-09-12T14:01:23.240 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37498 2020-09-12T14:01:23.240 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.240 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.240 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:01:23.240 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6807", 2020-09-12T14:01:23.240 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37498 2020-09-12T14:01:23.240 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.241 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.241 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.241 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:6803/37498", 2020-09-12T14:01:23.241 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addr": "127.0.0.1:6805/37498", 2020-09-12T14:01:23.241 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addr": "127.0.0.1:6809/37498", 2020-09-12T14:01:23.241 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addr": "127.0.0.1:6807/37498", 2020-09-12T14:01:23.241 INFO:tasks.workunit.client.0.smithi027.stdout: "state": [ 2020-09-12T14:01:23.242 INFO:tasks.workunit.client.0.smithi027.stdout: "exists", 2020-09-12T14:01:23.242 INFO:tasks.workunit.client.0.smithi027.stdout: "up" 2020-09-12T14:01:23.242 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.242 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.242 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.242 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:01:23.242 INFO:tasks.workunit.client.0.smithi027.stdout: "uuid": "d92b7341-d129-4e21-85d9-c431d9f9298f", 2020-09-12T14:01:23.242 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 0, 2020-09-12T14:01:23.243 INFO:tasks.workunit.client.0.smithi027.stdout: "in": 1, 2020-09-12T14:01:23.243 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 1, 2020-09-12T14:01:23.243 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_affinity": 1, 2020-09-12T14:01:23.243 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_begin": 0, 2020-09-12T14:01:23.243 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_end": 0, 2020-09-12T14:01:23.243 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 10, 2020-09-12T14:01:23.243 INFO:tasks.workunit.client.0.smithi027.stdout: "up_thru": 23, 2020-09-12T14:01:23.243 INFO:tasks.workunit.client.0.smithi027.stdout: "down_at": 31, 2020-09-12T14:01:23.244 INFO:tasks.workunit.client.0.smithi027.stdout: "lost_at": 0, 2020-09-12T14:01:23.244 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:01:23.244 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:01:23.244 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.244 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:01:23.244 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6810", 2020-09-12T14:01:23.244 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 32590 2020-09-12T14:01:23.245 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.245 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.245 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:01:23.245 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6811", 2020-09-12T14:01:23.245 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 32590 2020-09-12T14:01:23.245 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.245 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.246 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.246 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addrs": { 2020-09-12T14:01:23.246 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:01:23.246 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.246 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:01:23.246 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6812", 2020-09-12T14:01:23.246 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 32590 2020-09-12T14:01:23.247 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.247 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.247 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:01:23.247 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6813", 2020-09-12T14:01:23.247 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 32590 2020-09-12T14:01:23.247 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.247 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.247 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.248 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addrs": { 2020-09-12T14:01:23.248 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:01:23.248 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.248 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:01:23.248 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6816", 2020-09-12T14:01:23.248 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 32590 2020-09-12T14:01:23.249 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.249 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.249 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:01:23.249 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6817", 2020-09-12T14:01:23.249 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 32590 2020-09-12T14:01:23.249 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.249 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.250 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.250 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addrs": { 2020-09-12T14:01:23.250 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:01:23.250 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.250 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:01:23.250 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6814", 2020-09-12T14:01:23.251 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 32590 2020-09-12T14:01:23.251 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.251 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.251 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:01:23.251 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6815", 2020-09-12T14:01:23.251 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 32590 2020-09-12T14:01:23.251 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.251 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.252 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.252 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:6811/32590", 2020-09-12T14:01:23.252 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addr": "127.0.0.1:6813/32590", 2020-09-12T14:01:23.252 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addr": "127.0.0.1:6817/32590", 2020-09-12T14:01:23.252 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addr": "127.0.0.1:6815/32590", 2020-09-12T14:01:23.252 INFO:tasks.workunit.client.0.smithi027.stdout: "state": [ 2020-09-12T14:01:23.252 INFO:tasks.workunit.client.0.smithi027.stdout: "exists" 2020-09-12T14:01:23.253 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.253 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.253 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.253 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:01:23.253 INFO:tasks.workunit.client.0.smithi027.stdout: "uuid": "1b25b33b-61e6-4d39-ab58-fee24e94b77f", 2020-09-12T14:01:23.253 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 1, 2020-09-12T14:01:23.253 INFO:tasks.workunit.client.0.smithi027.stdout: "in": 1, 2020-09-12T14:01:23.253 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 1, 2020-09-12T14:01:23.254 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_affinity": 1, 2020-09-12T14:01:23.254 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_begin": 15, 2020-09-12T14:01:23.254 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_end": 29, 2020-09-12T14:01:23.254 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 36, 2020-09-12T14:01:23.254 INFO:tasks.workunit.client.0.smithi027.stdout: "up_thru": 0, 2020-09-12T14:01:23.254 INFO:tasks.workunit.client.0.smithi027.stdout: "down_at": 32, 2020-09-12T14:01:23.254 INFO:tasks.workunit.client.0.smithi027.stdout: "lost_at": 0, 2020-09-12T14:01:23.255 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:01:23.255 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:01:23.255 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.255 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:01:23.255 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6810", 2020-09-12T14:01:23.255 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37897 2020-09-12T14:01:23.255 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.256 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.256 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:01:23.256 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6811", 2020-09-12T14:01:23.256 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37897 2020-09-12T14:01:23.256 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.256 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.256 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.256 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addrs": { 2020-09-12T14:01:23.257 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:01:23.257 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.257 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:01:23.257 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6812", 2020-09-12T14:01:23.257 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37897 2020-09-12T14:01:23.257 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.257 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.257 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:01:23.258 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6813", 2020-09-12T14:01:23.258 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37897 2020-09-12T14:01:23.258 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.258 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.258 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.258 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addrs": { 2020-09-12T14:01:23.258 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:01:23.258 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.259 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:01:23.259 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6816", 2020-09-12T14:01:23.259 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37897 2020-09-12T14:01:23.259 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.259 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.259 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:01:23.259 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6817", 2020-09-12T14:01:23.260 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37897 2020-09-12T14:01:23.260 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.260 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.260 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.260 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addrs": { 2020-09-12T14:01:23.260 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:01:23.260 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.260 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:01:23.261 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6814", 2020-09-12T14:01:23.261 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37897 2020-09-12T14:01:23.261 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.261 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.261 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:01:23.261 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6815", 2020-09-12T14:01:23.261 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 37897 2020-09-12T14:01:23.262 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.262 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.262 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.262 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:6811/37897", 2020-09-12T14:01:23.262 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addr": "127.0.0.1:6813/37897", 2020-09-12T14:01:23.262 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addr": "127.0.0.1:6817/37897", 2020-09-12T14:01:23.262 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addr": "127.0.0.1:6815/37897", 2020-09-12T14:01:23.263 INFO:tasks.workunit.client.0.smithi027.stdout: "state": [ 2020-09-12T14:01:23.263 INFO:tasks.workunit.client.0.smithi027.stdout: "exists", 2020-09-12T14:01:23.263 INFO:tasks.workunit.client.0.smithi027.stdout: "up" 2020-09-12T14:01:23.263 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.263 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.263 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:01:23.263 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_xinfo": [ 2020-09-12T14:01:23.264 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.264 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:01:23.264 INFO:tasks.workunit.client.0.smithi027.stdout: "down_stamp": "2020-09-12T14:00:01.267275+0000", 2020-09-12T14:01:23.264 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_probability": 0, 2020-09-12T14:01:23.265 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_interval": 0, 2020-09-12T14:01:23.265 INFO:tasks.workunit.client.0.smithi027.stdout: "features": 4540138292841938943, 2020-09-12T14:01:23.265 INFO:tasks.workunit.client.0.smithi027.stdout: "old_weight": 0, 2020-09-12T14:01:23.265 INFO:tasks.workunit.client.0.smithi027.stdout: "last_purged_snaps_scrub": "2020-09-12T13:59:00.980797+0000", 2020-09-12T14:01:23.265 INFO:tasks.workunit.client.0.smithi027.stdout: "dead_epoch": 0 2020-09-12T14:01:23.265 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.266 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.266 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:01:23.266 INFO:tasks.workunit.client.0.smithi027.stdout: "down_stamp": "2020-09-12T14:00:01.870221+0000", 2020-09-12T14:01:23.266 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_probability": 0, 2020-09-12T14:01:23.266 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_interval": 0, 2020-09-12T14:01:23.266 INFO:tasks.workunit.client.0.smithi027.stdout: "features": 4540138292841938943, 2020-09-12T14:01:23.266 INFO:tasks.workunit.client.0.smithi027.stdout: "old_weight": 0, 2020-09-12T14:01:23.266 INFO:tasks.workunit.client.0.smithi027.stdout: "last_purged_snaps_scrub": "2020-09-12T13:59:08.852599+0000", 2020-09-12T14:01:23.267 INFO:tasks.workunit.client.0.smithi027.stdout: "dead_epoch": 0 2020-09-12T14:01:23.267 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.267 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.267 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:01:23.267 INFO:tasks.workunit.client.0.smithi027.stdout: "down_stamp": "2020-09-12T14:00:02.440346+0000", 2020-09-12T14:01:23.267 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_probability": 0, 2020-09-12T14:01:23.267 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_interval": 0, 2020-09-12T14:01:23.268 INFO:tasks.workunit.client.0.smithi027.stdout: "features": 4540138292841938943, 2020-09-12T14:01:23.268 INFO:tasks.workunit.client.0.smithi027.stdout: "old_weight": 0, 2020-09-12T14:01:23.268 INFO:tasks.workunit.client.0.smithi027.stdout: "last_purged_snaps_scrub": "2020-09-12T13:59:16.540209+0000", 2020-09-12T14:01:23.268 INFO:tasks.workunit.client.0.smithi027.stdout: "dead_epoch": 0 2020-09-12T14:01:23.268 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.268 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:01:23.268 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_upmap": [], 2020-09-12T14:01:23.268 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_upmap_items": [], 2020-09-12T14:01:23.269 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_temp": [], 2020-09-12T14:01:23.269 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_temp": [], 2020-09-12T14:01:23.269 INFO:tasks.workunit.client.0.smithi027.stdout: "blocklist": {}, 2020-09-12T14:01:23.269 INFO:tasks.workunit.client.0.smithi027.stdout: "erasure_code_profiles": { 2020-09-12T14:01:23.269 INFO:tasks.workunit.client.0.smithi027.stdout: "default": { 2020-09-12T14:01:23.269 INFO:tasks.workunit.client.0.smithi027.stdout: "k": "2", 2020-09-12T14:01:23.269 INFO:tasks.workunit.client.0.smithi027.stdout: "m": "2", 2020-09-12T14:01:23.269 INFO:tasks.workunit.client.0.smithi027.stdout: "plugin": "jerasure", 2020-09-12T14:01:23.270 INFO:tasks.workunit.client.0.smithi027.stdout: "technique": "reed_sol_van" 2020-09-12T14:01:23.270 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.270 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.270 INFO:tasks.workunit.client.0.smithi027.stdout: "removed_snaps_queue": [], 2020-09-12T14:01:23.270 INFO:tasks.workunit.client.0.smithi027.stdout: "new_removed_snaps": [], 2020-09-12T14:01:23.270 INFO:tasks.workunit.client.0.smithi027.stdout: "new_purged_snaps": [], 2020-09-12T14:01:23.270 INFO:tasks.workunit.client.0.smithi027.stdout: "crush_node_flags": {}, 2020-09-12T14:01:23.271 INFO:tasks.workunit.client.0.smithi027.stdout: "device_class_flags": {} 2020-09-12T14:01:23.271 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.271 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_metadata": [ 2020-09-12T14:01:23.271 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.271 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:01:23.271 INFO:tasks.workunit.client.0.smithi027.stdout: "arch": "x86_64", 2020-09-12T14:01:23.271 INFO:tasks.workunit.client.0.smithi027.stdout: "back_addr": "[v2:127.0.0.1:6804/37498,v1:127.0.0.1:6805/37498]", 2020-09-12T14:01:23.271 INFO:tasks.workunit.client.0.smithi027.stdout: "back_iface": "", 2020-09-12T14:01:23.272 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs": "1", 2020-09-12T14:01:23.272 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_db": "0", 2020-09-12T14:01:23.272 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_wal": "0", 2020-09-12T14:01:23.272 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_single_shared_device": "1", 2020-09-12T14:01:23.272 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_access_mode": "file", 2020-09-12T14:01:23.272 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_block_size": "4096", 2020-09-12T14:01:23.272 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_devices": "sda", 2020-09-12T14:01:23.272 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_driver": "KernelDevice", 2020-09-12T14:01:23.273 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2020-09-12T14:01:23.273 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_rotational": "1", 2020-09-12T14:01:23.273 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_size": "107374182400", 2020-09-12T14:01:23.273 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_support_discard": "0", 2020-09-12T14:01:23.273 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_type": "hdd", 2020-09-12T14:01:23.273 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_release": "pacific", 2020-09-12T14:01:23.273 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version": "ceph version 16.0.0-5485-gf346cc73d15 (f346cc73d15a568d3bec1c99a58953de76f57b85) pacific (dev)", 2020-09-12T14:01:23.274 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version_short": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:01:23.274 INFO:tasks.workunit.client.0.smithi027.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2020-09-12T14:01:23.274 INFO:tasks.workunit.client.0.smithi027.stdout: "default_device_class": "hdd", 2020-09-12T14:01:23.274 INFO:tasks.workunit.client.0.smithi027.stdout: "device_ids": "sda=ST1000NM0033-9ZM173_Z1W4H3N7", 2020-09-12T14:01:23.274 INFO:tasks.workunit.client.0.smithi027.stdout: "device_paths": "sda=/dev/disk/by-path/pci-0000:00:1f.2-ata-1", 2020-09-12T14:01:23.274 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": "sda", 2020-09-12T14:01:23.274 INFO:tasks.workunit.client.0.smithi027.stdout: "distro": "centos", 2020-09-12T14:01:23.274 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_description": "CentOS Linux 8 (Core)", 2020-09-12T14:01:23.275 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_version": "8", 2020-09-12T14:01:23.275 INFO:tasks.workunit.client.0.smithi027.stdout: "front_addr": "[v2:127.0.0.1:6802/37498,v1:127.0.0.1:6803/37498]", 2020-09-12T14:01:23.275 INFO:tasks.workunit.client.0.smithi027.stdout: "front_iface": "", 2020-09-12T14:01:23.275 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/37498,v1:127.0.0.1:6809/37498]", 2020-09-12T14:01:23.275 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/37498,v1:127.0.0.1:6807/37498]", 2020-09-12T14:01:23.275 INFO:tasks.workunit.client.0.smithi027.stdout: "hostname": "smithi027", 2020-09-12T14:01:23.275 INFO:tasks.workunit.client.0.smithi027.stdout: "journal_rotational": "1", 2020-09-12T14:01:23.276 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_description": "#1 SMP Sun Jul 26 03:54:29 UTC 2020", 2020-09-12T14:01:23.276 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_version": "4.18.0-193.14.2.el8_2.x86_64", 2020-09-12T14:01:23.276 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_swap_kb": "0", 2020-09-12T14:01:23.276 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_total_kb": "32659336", 2020-09-12T14:01:23.276 INFO:tasks.workunit.client.0.smithi027.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2020-09-12T14:01:23.276 INFO:tasks.workunit.client.0.smithi027.stdout: "objectstore_numa_unknown_devices": "sda", 2020-09-12T14:01:23.277 INFO:tasks.workunit.client.0.smithi027.stdout: "os": "Linux", 2020-09-12T14:01:23.277 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_data": "td/divergent-priors/0", 2020-09-12T14:01:23.277 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_objectstore": "bluestore", 2020-09-12T14:01:23.277 INFO:tasks.workunit.client.0.smithi027.stdout: "osdspec_affinity": "", 2020-09-12T14:01:23.277 INFO:tasks.workunit.client.0.smithi027.stdout: "rotational": "1" 2020-09-12T14:01:23.277 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.277 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.277 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:01:23.278 INFO:tasks.workunit.client.0.smithi027.stdout: "arch": "x86_64", 2020-09-12T14:01:23.278 INFO:tasks.workunit.client.0.smithi027.stdout: "back_addr": "[v2:127.0.0.1:6812/32590,v1:127.0.0.1:6813/32590]", 2020-09-12T14:01:23.278 INFO:tasks.workunit.client.0.smithi027.stdout: "back_iface": "", 2020-09-12T14:01:23.278 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs": "1", 2020-09-12T14:01:23.278 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_db": "0", 2020-09-12T14:01:23.278 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_wal": "0", 2020-09-12T14:01:23.279 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_single_shared_device": "1", 2020-09-12T14:01:23.279 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_access_mode": "file", 2020-09-12T14:01:23.279 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_block_size": "4096", 2020-09-12T14:01:23.279 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_devices": "sda", 2020-09-12T14:01:23.279 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_driver": "KernelDevice", 2020-09-12T14:01:23.279 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2020-09-12T14:01:23.279 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_rotational": "1", 2020-09-12T14:01:23.279 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_size": "107374182400", 2020-09-12T14:01:23.280 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_support_discard": "0", 2020-09-12T14:01:23.280 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_type": "hdd", 2020-09-12T14:01:23.280 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_release": "pacific", 2020-09-12T14:01:23.280 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version": "ceph version 16.0.0-5485-gf346cc73d15 (f346cc73d15a568d3bec1c99a58953de76f57b85) pacific (dev)", 2020-09-12T14:01:23.280 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version_short": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:01:23.280 INFO:tasks.workunit.client.0.smithi027.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2020-09-12T14:01:23.280 INFO:tasks.workunit.client.0.smithi027.stdout: "default_device_class": "hdd", 2020-09-12T14:01:23.281 INFO:tasks.workunit.client.0.smithi027.stdout: "device_ids": "sda=ST1000NM0033-9ZM173_Z1W4H3N7", 2020-09-12T14:01:23.281 INFO:tasks.workunit.client.0.smithi027.stdout: "device_paths": "sda=/dev/disk/by-path/pci-0000:00:1f.2-ata-1", 2020-09-12T14:01:23.281 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": "sda", 2020-09-12T14:01:23.281 INFO:tasks.workunit.client.0.smithi027.stdout: "distro": "centos", 2020-09-12T14:01:23.281 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_description": "CentOS Linux 8 (Core)", 2020-09-12T14:01:23.281 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_version": "8", 2020-09-12T14:01:23.281 INFO:tasks.workunit.client.0.smithi027.stdout: "front_addr": "[v2:127.0.0.1:6810/32590,v1:127.0.0.1:6811/32590]", 2020-09-12T14:01:23.281 INFO:tasks.workunit.client.0.smithi027.stdout: "front_iface": "", 2020-09-12T14:01:23.282 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/32590,v1:127.0.0.1:6817/32590]", 2020-09-12T14:01:23.282 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/32590,v1:127.0.0.1:6815/32590]", 2020-09-12T14:01:23.282 INFO:tasks.workunit.client.0.smithi027.stdout: "hostname": "smithi027", 2020-09-12T14:01:23.282 INFO:tasks.workunit.client.0.smithi027.stdout: "journal_rotational": "1", 2020-09-12T14:01:23.282 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_description": "#1 SMP Sun Jul 26 03:54:29 UTC 2020", 2020-09-12T14:01:23.282 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_version": "4.18.0-193.14.2.el8_2.x86_64", 2020-09-12T14:01:23.283 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_swap_kb": "0", 2020-09-12T14:01:23.283 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_total_kb": "32659336", 2020-09-12T14:01:23.283 INFO:tasks.workunit.client.0.smithi027.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2020-09-12T14:01:23.283 INFO:tasks.workunit.client.0.smithi027.stdout: "objectstore_numa_unknown_devices": "sda", 2020-09-12T14:01:23.283 INFO:tasks.workunit.client.0.smithi027.stdout: "os": "Linux", 2020-09-12T14:01:23.283 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_data": "td/divergent-priors/1", 2020-09-12T14:01:23.283 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_objectstore": "bluestore", 2020-09-12T14:01:23.283 INFO:tasks.workunit.client.0.smithi027.stdout: "osdspec_affinity": "", 2020-09-12T14:01:23.284 INFO:tasks.workunit.client.0.smithi027.stdout: "rotational": "1" 2020-09-12T14:01:23.284 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.284 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.284 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:01:23.284 INFO:tasks.workunit.client.0.smithi027.stdout: "arch": "x86_64", 2020-09-12T14:01:23.284 INFO:tasks.workunit.client.0.smithi027.stdout: "back_addr": "[v2:127.0.0.1:6812/37897,v1:127.0.0.1:6813/37897]", 2020-09-12T14:01:23.284 INFO:tasks.workunit.client.0.smithi027.stdout: "back_iface": "", 2020-09-12T14:01:23.285 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs": "1", 2020-09-12T14:01:23.285 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_db": "0", 2020-09-12T14:01:23.285 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_wal": "0", 2020-09-12T14:01:23.285 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_single_shared_device": "1", 2020-09-12T14:01:23.285 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_access_mode": "file", 2020-09-12T14:01:23.285 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_block_size": "4096", 2020-09-12T14:01:23.285 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_devices": "sda", 2020-09-12T14:01:23.286 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_driver": "KernelDevice", 2020-09-12T14:01:23.286 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2020-09-12T14:01:23.286 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_rotational": "1", 2020-09-12T14:01:23.286 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_size": "107374182400", 2020-09-12T14:01:23.286 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_support_discard": "0", 2020-09-12T14:01:23.286 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_type": "hdd", 2020-09-12T14:01:23.286 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_release": "pacific", 2020-09-12T14:01:23.286 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version": "ceph version 16.0.0-5485-gf346cc73d15 (f346cc73d15a568d3bec1c99a58953de76f57b85) pacific (dev)", 2020-09-12T14:01:23.287 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version_short": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:01:23.287 INFO:tasks.workunit.client.0.smithi027.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2020-09-12T14:01:23.287 INFO:tasks.workunit.client.0.smithi027.stdout: "default_device_class": "hdd", 2020-09-12T14:01:23.287 INFO:tasks.workunit.client.0.smithi027.stdout: "device_ids": "sda=ST1000NM0033-9ZM173_Z1W4H3N7", 2020-09-12T14:01:23.287 INFO:tasks.workunit.client.0.smithi027.stdout: "device_paths": "sda=/dev/disk/by-path/pci-0000:00:1f.2-ata-1", 2020-09-12T14:01:23.287 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": "sda", 2020-09-12T14:01:23.287 INFO:tasks.workunit.client.0.smithi027.stdout: "distro": "centos", 2020-09-12T14:01:23.287 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_description": "CentOS Linux 8 (Core)", 2020-09-12T14:01:23.288 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_version": "8", 2020-09-12T14:01:23.288 INFO:tasks.workunit.client.0.smithi027.stdout: "front_addr": "[v2:127.0.0.1:6810/37897,v1:127.0.0.1:6811/37897]", 2020-09-12T14:01:23.288 INFO:tasks.workunit.client.0.smithi027.stdout: "front_iface": "", 2020-09-12T14:01:23.288 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/37897,v1:127.0.0.1:6817/37897]", 2020-09-12T14:01:23.288 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/37897,v1:127.0.0.1:6815/37897]", 2020-09-12T14:01:23.288 INFO:tasks.workunit.client.0.smithi027.stdout: "hostname": "smithi027", 2020-09-12T14:01:23.288 INFO:tasks.workunit.client.0.smithi027.stdout: "journal_rotational": "1", 2020-09-12T14:01:23.289 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_description": "#1 SMP Sun Jul 26 03:54:29 UTC 2020", 2020-09-12T14:01:23.289 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_version": "4.18.0-193.14.2.el8_2.x86_64", 2020-09-12T14:01:23.289 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_swap_kb": "0", 2020-09-12T14:01:23.289 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_total_kb": "32659336", 2020-09-12T14:01:23.289 INFO:tasks.workunit.client.0.smithi027.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2020-09-12T14:01:23.289 INFO:tasks.workunit.client.0.smithi027.stdout: "objectstore_numa_unknown_devices": "sda", 2020-09-12T14:01:23.290 INFO:tasks.workunit.client.0.smithi027.stdout: "os": "Linux", 2020-09-12T14:01:23.290 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_data": "td/divergent-priors/2", 2020-09-12T14:01:23.290 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_objectstore": "bluestore", 2020-09-12T14:01:23.290 INFO:tasks.workunit.client.0.smithi027.stdout: "osdspec_affinity": "", 2020-09-12T14:01:23.290 INFO:tasks.workunit.client.0.smithi027.stdout: "rotational": "1" 2020-09-12T14:01:23.290 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.290 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:01:23.290 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap_clean_epochs": { 2020-09-12T14:01:23.291 INFO:tasks.workunit.client.0.smithi027.stdout: "min_last_epoch_clean": 0, 2020-09-12T14:01:23.291 INFO:tasks.workunit.client.0.smithi027.stdout: "last_epoch_clean": { 2020-09-12T14:01:23.291 INFO:tasks.workunit.client.0.smithi027.stdout: "per_pool": [] 2020-09-12T14:01:23.291 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.291 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_epochs": [ 2020-09-12T14:01:23.291 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.291 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:01:23.292 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 35 2020-09-12T14:01:23.292 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.292 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.292 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:01:23.292 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 10 2020-09-12T14:01:23.292 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.292 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.292 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:01:23.293 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 37 2020-09-12T14:01:23.293 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.293 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.293 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.293 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap_first_committed": 1, 2020-09-12T14:01:23.293 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap_last_committed": 39, 2020-09-12T14:01:23.293 INFO:tasks.workunit.client.0.smithi027.stdout: "crushmap": { 2020-09-12T14:01:23.293 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": [ 2020-09-12T14:01:23.294 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.294 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:01:23.294 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd.0", 2020-09-12T14:01:23.294 INFO:tasks.workunit.client.0.smithi027.stdout: "class": "hdd" 2020-09-12T14:01:23.294 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.294 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.294 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:01:23.295 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd.1", 2020-09-12T14:01:23.295 INFO:tasks.workunit.client.0.smithi027.stdout: "class": "hdd" 2020-09-12T14:01:23.295 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.295 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.295 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:01:23.295 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd.2", 2020-09-12T14:01:23.295 INFO:tasks.workunit.client.0.smithi027.stdout: "class": "hdd" 2020-09-12T14:01:23.296 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.296 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:01:23.296 INFO:tasks.workunit.client.0.smithi027.stdout: "types": [ 2020-09-12T14:01:23.296 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.296 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 0, 2020-09-12T14:01:23.296 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd" 2020-09-12T14:01:23.296 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.297 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.297 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 1, 2020-09-12T14:01:23.297 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "host" 2020-09-12T14:01:23.297 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.297 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.297 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 2, 2020-09-12T14:01:23.297 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "chassis" 2020-09-12T14:01:23.297 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.298 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.298 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 3, 2020-09-12T14:01:23.298 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "rack" 2020-09-12T14:01:23.298 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.298 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.298 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 4, 2020-09-12T14:01:23.298 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "row" 2020-09-12T14:01:23.299 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.299 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.299 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 5, 2020-09-12T14:01:23.299 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "pdu" 2020-09-12T14:01:23.299 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.299 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.299 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 6, 2020-09-12T14:01:23.299 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "pod" 2020-09-12T14:01:23.300 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.300 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.300 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 7, 2020-09-12T14:01:23.300 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "room" 2020-09-12T14:01:23.300 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.300 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.300 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 8, 2020-09-12T14:01:23.301 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "datacenter" 2020-09-12T14:01:23.301 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.301 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.301 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 9, 2020-09-12T14:01:23.301 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "zone" 2020-09-12T14:01:23.301 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.301 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.301 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 10, 2020-09-12T14:01:23.302 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "region" 2020-09-12T14:01:23.302 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.302 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.302 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 11, 2020-09-12T14:01:23.302 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "root" 2020-09-12T14:01:23.302 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.302 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:01:23.303 INFO:tasks.workunit.client.0.smithi027.stdout: "buckets": [ 2020-09-12T14:01:23.303 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.303 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -1, 2020-09-12T14:01:23.303 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "default", 2020-09-12T14:01:23.303 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 11, 2020-09-12T14:01:23.303 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "root", 2020-09-12T14:01:23.303 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:01:23.303 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:01:23.304 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:01:23.304 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:01:23.304 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.304 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -3, 2020-09-12T14:01:23.304 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:01:23.304 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:01:23.305 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.305 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.305 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.305 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.305 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -2, 2020-09-12T14:01:23.305 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "default~hdd", 2020-09-12T14:01:23.305 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 11, 2020-09-12T14:01:23.306 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "root", 2020-09-12T14:01:23.306 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:01:23.306 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:01:23.306 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:01:23.306 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:01:23.306 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.306 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -4, 2020-09-12T14:01:23.307 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:01:23.307 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:01:23.307 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.307 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.307 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.307 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.307 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -3, 2020-09-12T14:01:23.307 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "smithi027", 2020-09-12T14:01:23.308 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 1, 2020-09-12T14:01:23.308 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "host", 2020-09-12T14:01:23.308 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:01:23.308 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:01:23.308 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:01:23.308 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:01:23.308 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.308 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:01:23.309 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:01:23.309 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:01:23.309 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.309 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.309 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:01:23.309 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:01:23.309 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 1 2020-09-12T14:01:23.310 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.310 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.310 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:01:23.310 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:01:23.310 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 2 2020-09-12T14:01:23.310 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.310 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.310 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.311 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.311 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -4, 2020-09-12T14:01:23.311 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "smithi027~hdd", 2020-09-12T14:01:23.311 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 1, 2020-09-12T14:01:23.311 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "host", 2020-09-12T14:01:23.311 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:01:23.311 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:01:23.312 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:01:23.312 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:01:23.312 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.312 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:01:23.312 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:01:23.312 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:01:23.312 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.313 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.313 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:01:23.313 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:01:23.313 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 1 2020-09-12T14:01:23.313 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.313 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.313 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:01:23.314 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:01:23.314 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 2 2020-09-12T14:01:23.314 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.314 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.314 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.314 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:01:23.315 INFO:tasks.workunit.client.0.smithi027.stdout: "rules": [ 2020-09-12T14:01:23.315 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.315 INFO:tasks.workunit.client.0.smithi027.stdout: "rule_id": 0, 2020-09-12T14:01:23.315 INFO:tasks.workunit.client.0.smithi027.stdout: "rule_name": "replicated_rule", 2020-09-12T14:01:23.315 INFO:tasks.workunit.client.0.smithi027.stdout: "ruleset": 0, 2020-09-12T14:01:23.315 INFO:tasks.workunit.client.0.smithi027.stdout: "type": 1, 2020-09-12T14:01:23.315 INFO:tasks.workunit.client.0.smithi027.stdout: "min_size": 1, 2020-09-12T14:01:23.315 INFO:tasks.workunit.client.0.smithi027.stdout: "max_size": 10, 2020-09-12T14:01:23.316 INFO:tasks.workunit.client.0.smithi027.stdout: "steps": [ 2020-09-12T14:01:23.316 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.316 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "take", 2020-09-12T14:01:23.316 INFO:tasks.workunit.client.0.smithi027.stdout: "item": -1, 2020-09-12T14:01:23.316 INFO:tasks.workunit.client.0.smithi027.stdout: "item_name": "default" 2020-09-12T14:01:23.316 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.316 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.317 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "choose_firstn", 2020-09-12T14:01:23.317 INFO:tasks.workunit.client.0.smithi027.stdout: "num": 0, 2020-09-12T14:01:23.317 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "osd" 2020-09-12T14:01:23.317 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.317 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.317 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "emit" 2020-09-12T14:01:23.317 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.317 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:01:23.318 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.318 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:01:23.318 INFO:tasks.workunit.client.0.smithi027.stdout: "tunables": { 2020-09-12T14:01:23.318 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_local_tries": 0, 2020-09-12T14:01:23.318 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_local_fallback_tries": 0, 2020-09-12T14:01:23.318 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_total_tries": 50, 2020-09-12T14:01:23.318 INFO:tasks.workunit.client.0.smithi027.stdout: "chooseleaf_descend_once": 1, 2020-09-12T14:01:23.319 INFO:tasks.workunit.client.0.smithi027.stdout: "chooseleaf_vary_r": 1, 2020-09-12T14:01:23.319 INFO:tasks.workunit.client.0.smithi027.stdout: "chooseleaf_stable": 1, 2020-09-12T14:01:23.319 INFO:tasks.workunit.client.0.smithi027.stdout: "straw_calc_version": 1, 2020-09-12T14:01:23.319 INFO:tasks.workunit.client.0.smithi027.stdout: "allowed_bucket_algs": 54, 2020-09-12T14:01:23.319 INFO:tasks.workunit.client.0.smithi027.stdout: "profile": "jewel", 2020-09-12T14:01:23.319 INFO:tasks.workunit.client.0.smithi027.stdout: "optimal_tunables": 1, 2020-09-12T14:01:23.319 INFO:tasks.workunit.client.0.smithi027.stdout: "legacy_tunables": 0, 2020-09-12T14:01:23.320 INFO:tasks.workunit.client.0.smithi027.stdout: "minimum_required_version": "jewel", 2020-09-12T14:01:23.320 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables": 1, 2020-09-12T14:01:23.320 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables2": 1, 2020-09-12T14:01:23.320 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v2_rules": 0, 2020-09-12T14:01:23.320 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables3": 1, 2020-09-12T14:01:23.320 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v3_rules": 0, 2020-09-12T14:01:23.320 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v4_buckets": 1, 2020-09-12T14:01:23.320 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables5": 1, 2020-09-12T14:01:23.321 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v5_rules": 0 2020-09-12T14:01:23.321 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.321 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_args": {} 2020-09-12T14:01:23.321 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.321 INFO:tasks.workunit.client.0.smithi027.stdout: "fsmap": { 2020-09-12T14:01:23.321 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 1, 2020-09-12T14:01:23.321 INFO:tasks.workunit.client.0.smithi027.stdout: "default_fscid": -1, 2020-09-12T14:01:23.321 INFO:tasks.workunit.client.0.smithi027.stdout: "compat": { 2020-09-12T14:01:23.322 INFO:tasks.workunit.client.0.smithi027.stdout: "compat": {}, 2020-09-12T14:01:23.322 INFO:tasks.workunit.client.0.smithi027.stdout: "ro_compat": {}, 2020-09-12T14:01:23.322 INFO:tasks.workunit.client.0.smithi027.stdout: "incompat": { 2020-09-12T14:01:23.322 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_1": "base v0.20", 2020-09-12T14:01:23.322 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_2": "client writeable ranges", 2020-09-12T14:01:23.322 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_3": "default file layouts on dirs", 2020-09-12T14:01:23.322 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_4": "dir inode in separate object", 2020-09-12T14:01:23.323 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_5": "mds uses versioned encoding", 2020-09-12T14:01:23.323 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_6": "dirfrag is stored in omap", 2020-09-12T14:01:23.323 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_8": "no anchor table", 2020-09-12T14:01:23.323 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_9": "file layout v2", 2020-09-12T14:01:23.323 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_10": "snaprealm v2" 2020-09-12T14:01:23.323 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.323 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.323 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_flags": { 2020-09-12T14:01:23.324 INFO:tasks.workunit.client.0.smithi027.stdout: "enable_multiple": false, 2020-09-12T14:01:23.324 INFO:tasks.workunit.client.0.smithi027.stdout: "ever_enabled_multiple": false 2020-09-12T14:01:23.324 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.324 INFO:tasks.workunit.client.0.smithi027.stdout: "standbys": [], 2020-09-12T14:01:23.324 INFO:tasks.workunit.client.0.smithi027.stdout: "filesystems": [] 2020-09-12T14:01:23.324 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.325 INFO:tasks.workunit.client.0.smithi027.stdout: "mdsmap_first_committed": 1, 2020-09-12T14:01:23.325 INFO:tasks.workunit.client.0.smithi027.stdout: "mdsmap_last_committed": 1, 2020-09-12T14:01:23.325 INFO:tasks.workunit.client.0.smithi027.stdout: "auth": { 2020-09-12T14:01:23.325 INFO:tasks.workunit.client.0.smithi027.stdout: "first_committed": 1, 2020-09-12T14:01:23.325 INFO:tasks.workunit.client.0.smithi027.stdout: "last_committed": 5, 2020-09-12T14:01:23.325 INFO:tasks.workunit.client.0.smithi027.stdout: "num_secrets": 3 2020-09-12T14:01:23.325 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.325 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg": 1, 2020-09-12T14:01:23.326 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_active": 1, 2020-09-12T14:01:23.326 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_unknown": 0, 2020-09-12T14:01:23.326 INFO:tasks.workunit.client.0.smithi027.stdout: "num_osd": 3, 2020-09-12T14:01:23.326 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_sum": { 2020-09-12T14:01:23.326 INFO:tasks.workunit.client.0.smithi027.stdout: "stat_sum": { 2020-09-12T14:01:23.326 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes": 47947, 2020-09-12T14:01:23.326 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects": 100, 2020-09-12T14:01:23.327 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_clones": 0, 2020-09-12T14:01:23.327 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_copies": 300, 2020-09-12T14:01:23.327 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing_on_primary": 0, 2020-09-12T14:01:23.327 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing": 0, 2020-09-12T14:01:23.327 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_degraded": 100, 2020-09-12T14:01:23.327 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_misplaced": 0, 2020-09-12T14:01:23.327 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_unfound": 0, 2020-09-12T14:01:23.328 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_dirty": 100, 2020-09-12T14:01:23.328 INFO:tasks.workunit.client.0.smithi027.stdout: "num_whiteouts": 0, 2020-09-12T14:01:23.328 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read": 0, 2020-09-12T14:01:23.328 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read_kb": 0, 2020-09-12T14:01:23.328 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write": 101, 2020-09-12T14:01:23.328 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write_kb": 101, 2020-09-12T14:01:23.328 INFO:tasks.workunit.client.0.smithi027.stdout: "num_scrub_errors": 0, 2020-09-12T14:01:23.328 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shallow_scrub_errors": 0, 2020-09-12T14:01:23.329 INFO:tasks.workunit.client.0.smithi027.stdout: "num_deep_scrub_errors": 0, 2020-09-12T14:01:23.329 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_recovered": 0, 2020-09-12T14:01:23.329 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_recovered": 0, 2020-09-12T14:01:23.329 INFO:tasks.workunit.client.0.smithi027.stdout: "num_keys_recovered": 0, 2020-09-12T14:01:23.329 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_omap": 0, 2020-09-12T14:01:23.329 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_hit_set_archive": 0, 2020-09-12T14:01:23.329 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_hit_set_archive": 0, 2020-09-12T14:01:23.330 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush": 0, 2020-09-12T14:01:23.330 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_kb": 0, 2020-09-12T14:01:23.330 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict": 0, 2020-09-12T14:01:23.330 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_kb": 0, 2020-09-12T14:01:23.330 INFO:tasks.workunit.client.0.smithi027.stdout: "num_promote": 0, 2020-09-12T14:01:23.330 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_high": 0, 2020-09-12T14:01:23.330 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_low": 0, 2020-09-12T14:01:23.331 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_some": 0, 2020-09-12T14:01:23.331 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_full": 0, 2020-09-12T14:01:23.331 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_pinned": 0, 2020-09-12T14:01:23.331 INFO:tasks.workunit.client.0.smithi027.stdout: "num_legacy_snapsets": 0, 2020-09-12T14:01:23.331 INFO:tasks.workunit.client.0.smithi027.stdout: "num_large_omap_objects": 0, 2020-09-12T14:01:23.331 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_manifest": 0, 2020-09-12T14:01:23.331 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_bytes": 0, 2020-09-12T14:01:23.332 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_keys": 0, 2020-09-12T14:01:23.332 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_repaired": 0 2020-09-12T14:01:23.332 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.332 INFO:tasks.workunit.client.0.smithi027.stdout: "store_stats": { 2020-09-12T14:01:23.332 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 0, 2020-09-12T14:01:23.332 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 0, 2020-09-12T14:01:23.332 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:01:23.333 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 0, 2020-09-12T14:01:23.333 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 0, 2020-09-12T14:01:23.333 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:01:23.333 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:01:23.333 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:01:23.333 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 0, 2020-09-12T14:01:23.333 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 0 2020-09-12T14:01:23.334 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.334 INFO:tasks.workunit.client.0.smithi027.stdout: "log_size": 17, 2020-09-12T14:01:23.334 INFO:tasks.workunit.client.0.smithi027.stdout: "ondisk_log_size": 17, 2020-09-12T14:01:23.334 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 2, 2020-09-12T14:01:23.334 INFO:tasks.workunit.client.0.smithi027.stdout: "acting": 2, 2020-09-12T14:01:23.334 INFO:tasks.workunit.client.0.smithi027.stdout: "num_store_stats": 0 2020-09-12T14:01:23.334 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.334 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_sum": { 2020-09-12T14:01:23.335 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 0, 2020-09-12T14:01:23.335 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 0, 2020-09-12T14:01:23.335 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pgs": 3, 2020-09-12T14:01:23.335 INFO:tasks.workunit.client.0.smithi027.stdout: "num_osds": 3, 2020-09-12T14:01:23.335 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_osds": 3, 2020-09-12T14:01:23.335 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_omap_osds": 3, 2020-09-12T14:01:23.335 INFO:tasks.workunit.client.0.smithi027.stdout: "kb": 314572800, 2020-09-12T14:01:23.336 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used": 18168, 2020-09-12T14:01:23.336 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_data": 2072, 2020-09-12T14:01:23.336 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_omap": 1, 2020-09-12T14:01:23.336 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_meta": 16062, 2020-09-12T14:01:23.336 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_avail": 314554632, 2020-09-12T14:01:23.336 INFO:tasks.workunit.client.0.smithi027.stdout: "statfs": { 2020-09-12T14:01:23.336 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 322122547200, 2020-09-12T14:01:23.336 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 322103943168, 2020-09-12T14:01:23.337 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:01:23.337 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 2121728, 2020-09-12T14:01:23.337 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 402522, 2020-09-12T14:01:23.337 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:01:23.337 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:01:23.337 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:01:23.337 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 1928, 2020-09-12T14:01:23.338 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 16447608 2020-09-12T14:01:23.338 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.338 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_peers": [], 2020-09-12T14:01:23.338 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_trim_queue_len": 0, 2020-09-12T14:01:23.338 INFO:tasks.workunit.client.0.smithi027.stdout: "num_snap_trimming": 0, 2020-09-12T14:01:23.338 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shards_repaired": 0, 2020-09-12T14:01:23.339 INFO:tasks.workunit.client.0.smithi027.stdout: "op_queue_age_hist": { 2020-09-12T14:01:23.339 INFO:tasks.workunit.client.0.smithi027.stdout: "histogram": [], 2020-09-12T14:01:23.339 INFO:tasks.workunit.client.0.smithi027.stdout: "upper_bound": 1 2020-09-12T14:01:23.339 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.339 INFO:tasks.workunit.client.0.smithi027.stdout: "perf_stat": { 2020-09-12T14:01:23.339 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ms": 0, 2020-09-12T14:01:23.339 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ms": 0, 2020-09-12T14:01:23.339 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ns": 0, 2020-09-12T14:01:23.340 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ns": 0 2020-09-12T14:01:23.340 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.340 INFO:tasks.workunit.client.0.smithi027.stdout: "alerts": [], 2020-09-12T14:01:23.340 INFO:tasks.workunit.client.0.smithi027.stdout: "network_ping_times": [] 2020-09-12T14:01:23.340 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.340 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_sum_by_class": { 2020-09-12T14:01:23.341 INFO:tasks.workunit.client.0.smithi027.stdout: "hdd": { 2020-09-12T14:01:23.341 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 0, 2020-09-12T14:01:23.341 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 0, 2020-09-12T14:01:23.341 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pgs": 3, 2020-09-12T14:01:23.341 INFO:tasks.workunit.client.0.smithi027.stdout: "num_osds": 3, 2020-09-12T14:01:23.341 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_osds": 3, 2020-09-12T14:01:23.341 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_omap_osds": 3, 2020-09-12T14:01:23.342 INFO:tasks.workunit.client.0.smithi027.stdout: "kb": 314572800, 2020-09-12T14:01:23.342 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used": 18168, 2020-09-12T14:01:23.342 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_data": 2072, 2020-09-12T14:01:23.342 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_omap": 1, 2020-09-12T14:01:23.342 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_meta": 16062, 2020-09-12T14:01:23.342 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_avail": 314554632, 2020-09-12T14:01:23.342 INFO:tasks.workunit.client.0.smithi027.stdout: "statfs": { 2020-09-12T14:01:23.342 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 322122547200, 2020-09-12T14:01:23.343 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 322103943168, 2020-09-12T14:01:23.343 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:01:23.343 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 2121728, 2020-09-12T14:01:23.343 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 402522, 2020-09-12T14:01:23.343 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:01:23.343 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:01:23.343 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:01:23.344 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 1928, 2020-09-12T14:01:23.344 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 16447608 2020-09-12T14:01:23.344 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.344 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_peers": [], 2020-09-12T14:01:23.344 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_trim_queue_len": 0, 2020-09-12T14:01:23.344 INFO:tasks.workunit.client.0.smithi027.stdout: "num_snap_trimming": 0, 2020-09-12T14:01:23.344 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shards_repaired": 0, 2020-09-12T14:01:23.344 INFO:tasks.workunit.client.0.smithi027.stdout: "op_queue_age_hist": { 2020-09-12T14:01:23.345 INFO:tasks.workunit.client.0.smithi027.stdout: "histogram": [], 2020-09-12T14:01:23.345 INFO:tasks.workunit.client.0.smithi027.stdout: "upper_bound": 1 2020-09-12T14:01:23.345 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.345 INFO:tasks.workunit.client.0.smithi027.stdout: "perf_stat": { 2020-09-12T14:01:23.345 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ms": 0, 2020-09-12T14:01:23.345 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ms": 0, 2020-09-12T14:01:23.345 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ns": 0, 2020-09-12T14:01:23.345 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ns": 0 2020-09-12T14:01:23.346 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.346 INFO:tasks.workunit.client.0.smithi027.stdout: "alerts": [], 2020-09-12T14:01:23.346 INFO:tasks.workunit.client.0.smithi027.stdout: "network_ping_times": [] 2020-09-12T14:01:23.346 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.346 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.346 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_stats": [ 2020-09-12T14:01:23.346 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.346 INFO:tasks.workunit.client.0.smithi027.stdout: "poolid": 1, 2020-09-12T14:01:23.347 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg": 1, 2020-09-12T14:01:23.347 INFO:tasks.workunit.client.0.smithi027.stdout: "stat_sum": { 2020-09-12T14:01:23.347 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes": 47947, 2020-09-12T14:01:23.347 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects": 100, 2020-09-12T14:01:23.347 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_clones": 0, 2020-09-12T14:01:23.347 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_copies": 300, 2020-09-12T14:01:23.347 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing_on_primary": 0, 2020-09-12T14:01:23.348 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing": 0, 2020-09-12T14:01:23.348 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_degraded": 100, 2020-09-12T14:01:23.348 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_misplaced": 0, 2020-09-12T14:01:23.348 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_unfound": 0, 2020-09-12T14:01:23.348 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_dirty": 100, 2020-09-12T14:01:23.348 INFO:tasks.workunit.client.0.smithi027.stdout: "num_whiteouts": 0, 2020-09-12T14:01:23.348 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read": 0, 2020-09-12T14:01:23.349 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read_kb": 0, 2020-09-12T14:01:23.349 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write": 101, 2020-09-12T14:01:23.349 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write_kb": 101, 2020-09-12T14:01:23.349 INFO:tasks.workunit.client.0.smithi027.stdout: "num_scrub_errors": 0, 2020-09-12T14:01:23.349 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shallow_scrub_errors": 0, 2020-09-12T14:01:23.349 INFO:tasks.workunit.client.0.smithi027.stdout: "num_deep_scrub_errors": 0, 2020-09-12T14:01:23.349 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_recovered": 0, 2020-09-12T14:01:23.349 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_recovered": 0, 2020-09-12T14:01:23.350 INFO:tasks.workunit.client.0.smithi027.stdout: "num_keys_recovered": 0, 2020-09-12T14:01:23.350 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_omap": 0, 2020-09-12T14:01:23.350 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_hit_set_archive": 0, 2020-09-12T14:01:23.350 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_hit_set_archive": 0, 2020-09-12T14:01:23.350 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush": 0, 2020-09-12T14:01:23.350 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_kb": 0, 2020-09-12T14:01:23.350 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict": 0, 2020-09-12T14:01:23.350 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_kb": 0, 2020-09-12T14:01:23.351 INFO:tasks.workunit.client.0.smithi027.stdout: "num_promote": 0, 2020-09-12T14:01:23.351 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_high": 0, 2020-09-12T14:01:23.351 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_low": 0, 2020-09-12T14:01:23.351 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_some": 0, 2020-09-12T14:01:23.351 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_full": 0, 2020-09-12T14:01:23.351 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_pinned": 0, 2020-09-12T14:01:23.351 INFO:tasks.workunit.client.0.smithi027.stdout: "num_legacy_snapsets": 0, 2020-09-12T14:01:23.351 INFO:tasks.workunit.client.0.smithi027.stdout: "num_large_omap_objects": 0, 2020-09-12T14:01:23.352 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_manifest": 0, 2020-09-12T14:01:23.352 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_bytes": 0, 2020-09-12T14:01:23.352 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_keys": 0, 2020-09-12T14:01:23.352 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_repaired": 0 2020-09-12T14:01:23.352 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.352 INFO:tasks.workunit.client.0.smithi027.stdout: "store_stats": { 2020-09-12T14:01:23.352 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 0, 2020-09-12T14:01:23.353 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 0, 2020-09-12T14:01:23.353 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:01:23.353 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 1220608, 2020-09-12T14:01:23.353 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 142522, 2020-09-12T14:01:23.353 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:01:23.353 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:01:23.353 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:01:23.354 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 0, 2020-09-12T14:01:23.354 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 0 2020-09-12T14:01:23.354 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.354 INFO:tasks.workunit.client.0.smithi027.stdout: "log_size": 17, 2020-09-12T14:01:23.354 INFO:tasks.workunit.client.0.smithi027.stdout: "ondisk_log_size": 17, 2020-09-12T14:01:23.354 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 2, 2020-09-12T14:01:23.354 INFO:tasks.workunit.client.0.smithi027.stdout: "acting": 2, 2020-09-12T14:01:23.355 INFO:tasks.workunit.client.0.smithi027.stdout: "num_store_stats": 3 2020-09-12T14:01:23.355 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.355 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:01:23.355 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_stats": [ 2020-09-12T14:01:23.355 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.355 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:01:23.355 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 150323855380 2020-09-12T14:01:23.356 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.356 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.356 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:01:23.356 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 42949672977 2020-09-12T14:01:23.356 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.356 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.356 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:01:23.356 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 154618822675 2020-09-12T14:01:23.357 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.357 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:01:23.357 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_by_state": [ 2020-09-12T14:01:23.357 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.357 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "active+undersized+degraded", 2020-09-12T14:01:23.357 INFO:tasks.workunit.client.0.smithi027.stdout: "num": 1 2020-09-12T14:01:23.357 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.358 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:01:23.358 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_by_osd": [ 2020-09-12T14:01:23.358 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.358 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:01:23.358 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 0, 2020-09-12T14:01:23.358 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 1, 2020-09-12T14:01:23.358 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T14:01:23.358 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.359 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.359 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:01:23.359 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 0, 2020-09-12T14:01:23.359 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 0, 2020-09-12T14:01:23.359 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T14:01:23.359 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.359 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.360 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:01:23.360 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 1, 2020-09-12T14:01:23.360 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 1, 2020-09-12T14:01:23.360 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T14:01:23.360 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.360 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:01:23.360 INFO:tasks.workunit.client.0.smithi027.stdout: "purged_snaps": [ 2020-09-12T14:01:23.360 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:01:23.361 INFO:tasks.workunit.client.0.smithi027.stdout: "pool": 1, 2020-09-12T14:01:23.361 INFO:tasks.workunit.client.0.smithi027.stdout: "purged_snaps": {} 2020-09-12T14:01:23.361 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.361 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:01:23.361 INFO:tasks.workunit.client.0.smithi027.stdout: "servicemap": { 2020-09-12T14:01:23.361 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 6, 2020-09-12T14:01:23.361 INFO:tasks.workunit.client.0.smithi027.stdout: "modified": "2020-09-12T14:01:08.738308+0000", 2020-09-12T14:01:23.362 INFO:tasks.workunit.client.0.smithi027.stdout: "services": {} 2020-09-12T14:01:23.362 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:01:23.362 INFO:tasks.workunit.client.0.smithi027.stdout: "paxos": { 2020-09-12T14:01:23.362 INFO:tasks.workunit.client.0.smithi027.stdout: "first_committed": 1, 2020-09-12T14:01:23.362 INFO:tasks.workunit.client.0.smithi027.stdout: "last_committed": 440, 2020-09-12T14:01:23.362 INFO:tasks.workunit.client.0.smithi027.stdout: "last_pn": 0, 2020-09-12T14:01:23.362 INFO:tasks.workunit.client.0.smithi027.stdout: "accepted_pn": 0 2020-09-12T14:01:23.363 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:01:23.363 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:01:23.363 INFO:tasks.workunit.client.0.smithi027.stderr:report 2657114333 2020-09-12T14:01:23.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1604: wait_for_clean: return 1 2020-09-12T14:01:23.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:536: TEST_divergent_2: echo 'delay recovery' 2020-09-12T14:01:23.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:537: TEST_divergent_2: ceph pg dump pgs 2020-09-12T14:01:23.364 INFO:tasks.workunit.client.0.smithi027.stdout:delay recovery 2020-09-12T14:01:23.596 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:01:23.597 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 0 100 0 0 47947 0 0 17 17 active+undersized+degraded 2020-09-12T14:00:10.331732+0000 39'101 39:121 [0,2] 0 [0,2] 0 0'0 2020-09-12T13:59:19.138081+0000 0'0 2020-09-12T13:59:19.138081+0000 0 2020-09-12T14:01:23.597 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:01:23.597 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:01:23.598 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:01:23.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:538: TEST_divergent_2: for i in $non_divergent 2020-09-12T14:01:23.608 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: get_asok_path osd.0 2020-09-12T14:01:23.608 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2020-09-12T14:01:23.608 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2020-09-12T14:01:23.609 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:01:23.609 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:01:23.609 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:01:23.609 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.0.asok 2020-09-12T14:01:23.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: CEPH_ARGS= 2020-09-12T14:01:23.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.0.asok set_recovery_delay 100000 2020-09-12T14:01:23.716 INFO:tasks.workunit.client.0.smithi027.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2020-09-12T14:01:23.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:538: TEST_divergent_2: for i in $non_divergent 2020-09-12T14:01:23.725 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: get_asok_path osd.2 2020-09-12T14:01:23.726 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2020-09-12T14:01:23.726 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2020-09-12T14:01:23.726 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:01:23.726 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:01:23.726 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:01:23.727 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.2.asok 2020-09-12T14:01:23.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: CEPH_ARGS= 2020-09-12T14:01:23.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.2.asok set_recovery_delay 100000 2020-09-12T14:01:23.833 INFO:tasks.workunit.client.0.smithi027.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2020-09-12T14:01:23.841 INFO:tasks.workunit.client.0.smithi027.stdout:revive divergent 1 2020-09-12T14:01:23.841 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:544: TEST_divergent_2: echo 'revive divergent 1' 2020-09-12T14:01:23.841 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:545: TEST_divergent_2: ceph pg dump pgs 2020-09-12T14:01:24.224 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:01:24.225 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 0 100 0 0 47947 0 0 17 17 active+undersized+degraded 2020-09-12T14:00:10.331732+0000 39'101 39:121 [0,2] 0 [0,2] 0 0'0 2020-09-12T13:59:19.138081+0000 0'0 2020-09-12T13:59:19.138081+0000 0 2020-09-12T14:01:24.225 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:01:24.225 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:01:24.226 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:01:24.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:546: TEST_divergent_2: ceph osd set noup 2020-09-12T14:01:24.959 INFO:tasks.workunit.client.0.smithi027.stderr:noup is set 2020-09-12T14:01:24.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:547: TEST_divergent_2: activate_osd td/divergent-priors 1 2020-09-12T14:01:24.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/divergent-priors 2020-09-12T14:01:24.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T14:01:24.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=1 2020-09-12T14:01:24.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T14:01:24.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/divergent-priors/1 2020-09-12T14:01:24.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T14:01:24.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:01:24.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:01:24.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:01:24.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2020-09-12T14:01:24.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2020-09-12T14:01:24.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T14:01:24.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T14:01:24.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T14:01:24.973 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T14:01:24.973 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:01:24.973 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:01:24.973 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:01:24.973 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:01:24.973 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:01:24.974 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:01:24.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:01:24.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:01:24.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T14:01:24.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T14:01:24.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:01:24.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:01:24.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:01:24.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T14:01:24.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T14:01:24.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/divergent-priors/1 2020-09-12T14:01:24.976 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T14:01:24.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.1 2020-09-12T14:01:24.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 1 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:01:24.977 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/divergent-priors/1/whoami 2020-09-12T14:01:24.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 1 = 1 ']' 2020-09-12T14:01:24.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T14:01:24.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T14:01:24.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T14:01:25.000 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:01:24.998+0000 7f0b01e29ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:01:25.001 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:01:25.000+0000 7f0b01e29ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:01:25.003 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:01:25.001+0000 7f0b01e29ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:01:25.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:877: activate_osd: return 0 2020-09-12T14:01:25.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:548: TEST_divergent_2: sleep 5 2020-09-12T14:01:25.522 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:01:25.520+0000 7f0b01e29ec0 -1 Falling back to public interface 2020-09-12T14:01:26.789 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:01:26.787+0000 7f0b01e29ec0 -1 osd.1 29 log_to_monitors {default=true} 2020-09-12T14:01:30.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:550: TEST_divergent_2: echo 'delay recovery divergent' 2020-09-12T14:01:30.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:551: TEST_divergent_2: ceph pg dump pgs 2020-09-12T14:01:30.393 INFO:tasks.workunit.client.0.smithi027.stdout:delay recovery divergent 2020-09-12T14:01:30.787 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:01:30.787 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 0 100 0 0 47947 0 0 17 17 active+undersized+degraded 2020-09-12T14:00:10.331732+0000 39'101 39:122 [0,2] 0 [0,2] 0 0'0 2020-09-12T13:59:19.138081+0000 0'0 2020-09-12T13:59:19.138081+0000 0 2020-09-12T14:01:30.788 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:01:30.788 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:01:30.789 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:01:30.799 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:552: TEST_divergent_2: get_asok_path osd.1 2020-09-12T14:01:30.799 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2020-09-12T14:01:30.799 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2020-09-12T14:01:30.799 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:01:30.799 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:01:30.799 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:01:30.800 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.1.asok 2020-09-12T14:01:30.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:552: TEST_divergent_2: CEPH_ARGS= 2020-09-12T14:01:30.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:552: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.1.asok set_recovery_delay 100000 2020-09-12T14:01:30.906 INFO:tasks.workunit.client.0.smithi027.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2020-09-12T14:01:30.914 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:554: TEST_divergent_2: ceph osd unset noup 2020-09-12T14:01:31.636 INFO:tasks.workunit.client.0.smithi027.stderr:noup is unset 2020-09-12T14:01:31.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:556: TEST_divergent_2: wait_for_osd up 0 2020-09-12T14:01:31.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:01:31.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T14:01:31.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:01:31.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:01:31.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:01:31.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:01:31.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:01:31.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:01:31.647 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:01:32.080 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 35 up_thru 38 down_at 30 last_clean_interval [5,29) [v2:127.0.0.1:6802/37498,v1:127.0.0.1:6803/37498] [v2:127.0.0.1:6804/37498,v1:127.0.0.1:6805/37498] exists,up 6e938613-9665-406c-b148-ce5bfd24c111 2020-09-12T14:01:32.080 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:01:32.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:01:32.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:01:32.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:01:32.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:557: TEST_divergent_2: wait_for_osd up 1 2020-09-12T14:01:32.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:01:32.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:01:32.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:01:32.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:01:32.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:01:32.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:01:32.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:01:32.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:01:32.499 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 43 up_thru 43 down_at 31 last_clean_interval [10,29) [v2:127.0.0.1:6818/40417,v1:127.0.0.1:6819/40417] [v2:127.0.0.1:6820/40417,v1:127.0.0.1:6821/40417] exists,up d92b7341-d129-4e21-85d9-c431d9f9298f 2020-09-12T14:01:32.500 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:01:32.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:01:32.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:01:32.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:01:32.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:558: TEST_divergent_2: wait_for_osd up 2 2020-09-12T14:01:32.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:01:32.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T14:01:32.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:01:32.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:01:32.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:01:32.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:01:32.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:01:32.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:01:32.907 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up in weight 1 up_from 36 up_thru 0 down_at 32 last_clean_interval [15,29) [v2:127.0.0.1:6810/37897,v1:127.0.0.1:6811/37897] [v2:127.0.0.1:6812/37897,v1:127.0.0.1:6813/37897] exists,up 1b25b33b-61e6-4d39-ab58-fee24e94b77f 2020-09-12T14:01:32.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:01:32.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:01:32.908 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:01:32.908 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:560: TEST_divergent_2: ceph pg dump pgs 2020-09-12T14:01:33.326 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:01:33.327 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 0 0 0 0 46628 0 0 16 16 peering 2020-09-12T14:01:31.785179+0000 29'106 43:116 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T13:59:19.138081+0000 0'0 2020-09-12T13:59:19.138081+0000 0 2020-09-12T14:01:33.327 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:01:33.327 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:01:33.328 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:01:33.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:561: TEST_divergent_2: echo 'wait for peering' 2020-09-12T14:01:33.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:562: TEST_divergent_2: ceph pg dump pgs 2020-09-12T14:01:33.338 INFO:tasks.workunit.client.0.smithi027.stdout:wait for peering 2020-09-12T14:01:33.718 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:01:33.719 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 0 0 0 0 46628 0 0 16 16 peering 2020-09-12T14:01:31.785179+0000 29'106 43:116 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T13:59:19.138081+0000 0'0 2020-09-12T13:59:19.138081+0000 0 2020-09-12T14:01:33.719 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:01:33.719 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:01:33.720 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:01:33.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:563: TEST_divergent_2: rados -p test put foo /etc/fstab 2020-09-12T14:01:46.445 INFO:tasks.workunit.client.0.smithi027.stdout:killing divergent 1 2020-09-12T14:01:46.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:567: TEST_divergent_2: echo 'killing divergent 1' 2020-09-12T14:01:46.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:568: TEST_divergent_2: ceph pg dump pgs 2020-09-12T14:01:46.838 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:01:46.839 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 4 4 0 0 47947 0 0 11 11 active+recovering+degraded+wait 2020-09-12T14:01:32.387999+0000 39'101 44:125 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T13:59:19.138081+0000 0'0 2020-09-12T13:59:19.138081+0000 0 2020-09-12T14:01:46.839 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:01:46.839 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:01:46.840 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:01:46.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:569: TEST_divergent_2: kill_daemons td/divergent-priors KILL osd.1 2020-09-12T14:01:46.850 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:01:46.850 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:01:46.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:01:46.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:01:46.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:01:46.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:01:46.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:572: TEST_divergent_2: expfile=td/divergent-priors/exp.20815.out 2020-09-12T14:01:46.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:573: TEST_divergent_2: _objectstore_tool_nodown td/divergent-priors 1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.20815.out 2020-09-12T14:01:46.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1195: _objectstore_tool_nodown: local dir=td/divergent-priors 2020-09-12T14:01:46.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: _objectstore_tool_nodown: shift 2020-09-12T14:01:46.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: _objectstore_tool_nodown: local id=1 2020-09-12T14:01:46.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1198: _objectstore_tool_nodown: shift 2020-09-12T14:01:46.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2020-09-12T14:01:46.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1202: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.20815.out 2020-09-12T14:01:47.637 INFO:tasks.workunit.client.0.smithi027.stderr:Exporting 1.0 info 1.0( v 44'102 lc 0'0 (29'90,44'102] local-lis/les=43/44 n=101 ec=23/23 lis/c=43/23 les/c/f=44/24/0 sis=43 pruub=13.947920054s) 2020-09-12T14:01:47.651 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:023dd3d2:::existing_12:head# 2020-09-12T14:01:47.652 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:04c40a0c:::existing_93:head# 2020-09-12T14:01:47.652 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:064ff5fb:::existing_36:head# 2020-09-12T14:01:47.652 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:091f9f1d:::existing_94:head# 2020-09-12T14:01:47.654 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:0a865a93:::existing_54:head# 2020-09-12T14:01:47.655 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:0fed513c:::existing_41:head# 2020-09-12T14:01:47.655 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:13b9e0b8:::existing_92:head# 2020-09-12T14:01:47.656 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:169157fa:::existing_81:head# 2020-09-12T14:01:47.656 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:17731382:::existing_58:head# 2020-09-12T14:01:47.656 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:18a45a39:::existing_43:head# 2020-09-12T14:01:47.657 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:1c160f01:::existing_66:head# 2020-09-12T14:01:47.657 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:1dd6c249:::existing_97:head# 2020-09-12T14:01:47.657 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:227e0c77:::existing_7:head# 2020-09-12T14:01:47.658 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:23e1b063:::existing_59:head# 2020-09-12T14:01:47.658 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:28566edb:::existing_78:head# 2020-09-12T14:01:47.659 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:2a101182:::existing_20:head# 2020-09-12T14:01:47.659 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:2d1519da:::existing_96:head# 2020-09-12T14:01:47.659 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:3078b87a:::existing_86:head# 2020-09-12T14:01:47.661 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:30bde01b:::existing_15:head# 2020-09-12T14:01:47.661 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:317a7e19:::existing_56:head# 2020-09-12T14:01:47.661 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:38b9c7fa:::existing_33:head# 2020-09-12T14:01:47.662 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:3e16d84b:::existing_48:head# 2020-09-12T14:01:47.662 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:45fe8203:::existing_75:head# 2020-09-12T14:01:47.662 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:4e82c9ba:::existing_50:head# 2020-09-12T14:01:47.662 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5298dcdb:::existing_57:head# 2020-09-12T14:01:47.662 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:52ebaa44:::existing_32:head# 2020-09-12T14:01:47.662 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:53588912:::existing_9:head# 2020-09-12T14:01:47.663 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5b7e2de6:::existing_84:head# 2020-09-12T14:01:47.663 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5da0255c:::existing_13:head# 2020-09-12T14:01:47.663 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5da2dab8:::existing_53:head# 2020-09-12T14:01:47.663 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5ee3514b:::existing_30:head# 2020-09-12T14:01:47.663 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:602f83fe:::foo:head# 2020-09-12T14:01:47.663 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:60426cc7:::existing_62:head# 2020-09-12T14:01:47.664 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:628918ca:::existing_79:head# 2020-09-12T14:01:47.664 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:63f8ab2d:::existing_77:head# 2020-09-12T14:01:47.665 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:67e1780f:::existing_27:head# 2020-09-12T14:01:47.665 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:68143ca1:::existing_71:head# 2020-09-12T14:01:47.666 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:6bac9509:::existing_29:head# 2020-09-12T14:01:47.668 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:6bbec798:::existing_45:head# 2020-09-12T14:01:47.668 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:6cb23a40:::existing_65:head# 2020-09-12T14:01:47.669 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7055ec5e:::existing_83:head# 2020-09-12T14:01:47.672 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:70a54e2b:::existing_87:head# 2020-09-12T14:01:47.672 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:750ace21:::existing_19:head# 2020-09-12T14:01:47.672 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:78623b1d:::existing_6:head# 2020-09-12T14:01:47.672 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:79881152:::existing_63:head# 2020-09-12T14:01:47.672 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7a80c0d5:::existing_99:head# 2020-09-12T14:01:47.674 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:81490677:::existing_76:head# 2020-09-12T14:01:47.674 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:85ec8e5d:::existing_85:head# 2020-09-12T14:01:47.674 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8e2abbf0:::existing_49:head# 2020-09-12T14:01:47.674 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8ee0fe0f:::existing_52:head# 2020-09-12T14:01:47.674 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:923c5d4d:::existing_5:head# 2020-09-12T14:01:47.674 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:92731e71:::existing_89:head# 2020-09-12T14:01:47.675 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:928205cf:::existing_82:head# 2020-09-12T14:01:47.675 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:93be2267:::existing_24:head# 2020-09-12T14:01:47.675 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:99ceda2c:::existing_44:head# 2020-09-12T14:01:47.676 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:9dd917ee:::existing_26:head# 2020-09-12T14:01:47.676 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a28805a6:::existing_72:head# 2020-09-12T14:01:47.677 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a294a8f6:::existing_16:head# 2020-09-12T14:01:47.677 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a2f5473a:::existing_42:head# 2020-09-12T14:01:47.677 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a504a533:::existing_90:head# 2020-09-12T14:01:47.677 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a691337e:::existing_61:head# 2020-09-12T14:01:47.677 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a6e8d7ac:::existing_28:head# 2020-09-12T14:01:47.677 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a9fde86e:::existing_18:head# 2020-09-12T14:01:47.677 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:aa309faa:::existing_68:head# 2020-09-12T14:01:47.678 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ae40b0ed:::existing_31:head# 2020-09-12T14:01:47.678 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b279b1fe:::existing_10:head# 2020-09-12T14:01:47.678 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b343e277:::existing_88:head# 2020-09-12T14:01:47.682 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b7130e90:::existing_80:head# 2020-09-12T14:01:47.683 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:bc6f7309:::existing_70:head# 2020-09-12T14:01:47.686 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:be8489b7:::existing_38:head# 2020-09-12T14:01:47.686 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:beb4e5e3:::existing_47:head# 2020-09-12T14:01:47.686 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:bebec877:::existing_74:head# 2020-09-12T14:01:47.686 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c04347d8:::existing_60:head# 2020-09-12T14:01:47.687 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c156033e:::existing_100:head# 2020-09-12T14:01:47.687 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c19f5a8e:::existing_8:head# 2020-09-12T14:01:47.687 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c3e6ebe4:::existing_17:head# 2020-09-12T14:01:47.687 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c6c5a7c2:::existing_39:head# 2020-09-12T14:01:47.687 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:cc1d0abf:::existing_95:head# 2020-09-12T14:01:47.687 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ccb3865b:::existing_73:head# 2020-09-12T14:01:47.687 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:cd84e19e:::existing_40:head# 2020-09-12T14:01:47.688 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d2cd64ce:::existing_37:head# 2020-09-12T14:01:47.688 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d41a25c9:::existing_22:head# 2020-09-12T14:01:47.688 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d7f088da:::existing_91:head# 2020-09-12T14:01:47.690 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d9cbd1d6:::existing_64:head# 2020-09-12T14:01:47.690 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:dcc4ff82:::existing_98:head# 2020-09-12T14:01:47.690 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:dd677bd3:::existing_35:head# 2020-09-12T14:01:47.691 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:dd6d90e7:::existing_46:head# 2020-09-12T14:01:47.694 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:e0e22077:::existing_21:head# 2020-09-12T14:01:47.694 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:e255b710:::existing_55:head# 2020-09-12T14:01:47.695 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:eff1d5db:::existing_14:head# 2020-09-12T14:01:47.695 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:f1b156bf:::existing_23:head# 2020-09-12T14:01:47.695 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:f5ce4199:::existing_34:head# 2020-09-12T14:01:47.695 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:f6809456:::existing_11:head# 2020-09-12T14:01:47.695 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:f699a2a9:::existing_67:head# 2020-09-12T14:01:47.695 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:f9baac43:::existing_25:head# 2020-09-12T14:01:47.695 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:fb76688e:::existing_51:head# 2020-09-12T14:01:47.696 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:fbb92eec:::existing_69:head# 2020-09-12T14:01:47.696 INFO:tasks.workunit.client.0.smithi027.stderr:Export successful 2020-09-12T14:01:47.696 INFO:tasks.workunit.client.0.smithi027.stdout: marking collection for removal 2020-09-12T14:01:47.696 INFO:tasks.workunit.client.0.smithi027.stdout:setting '_remove' omap key 2020-09-12T14:01:47.696 INFO:tasks.workunit.client.0.smithi027.stdout:finish_remove_pgs 1.0_head removing 1.0 2020-09-12T14:01:47.880 INFO:tasks.workunit.client.0.smithi027.stderr:Remove successful 2020-09-12T14:01:48.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:574: TEST_divergent_2: _objectstore_tool_nodown td/divergent-priors 1 --op import --file td/divergent-priors/exp.20815.out 2020-09-12T14:01:48.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1195: _objectstore_tool_nodown: local dir=td/divergent-priors 2020-09-12T14:01:48.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: _objectstore_tool_nodown: shift 2020-09-12T14:01:48.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: _objectstore_tool_nodown: local id=1 2020-09-12T14:01:48.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1198: _objectstore_tool_nodown: shift 2020-09-12T14:01:48.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2020-09-12T14:01:48.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1202: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op import --file td/divergent-priors/exp.20815.out 2020-09-12T14:01:49.037 INFO:tasks.workunit.client.0.smithi027.stdout:get_pg_num_history pg_num_history pg_num_history(e44 pg_nums {1={23=1}} deleted_pools ) 2020-09-12T14:01:49.037 INFO:tasks.workunit.client.0.smithi027.stdout:Importing pgid 1.0 2020-09-12T14:01:49.038 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:023dd3d2:::existing_12:head# 2020-09-12T14:01:49.038 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:49.175 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:04c40a0c:::existing_93:head# 2020-09-12T14:01:49.176 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:49.216 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:064ff5fb:::existing_36:head# 2020-09-12T14:01:49.216 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:49.249 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:091f9f1d:::existing_94:head# 2020-09-12T14:01:49.250 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:49.283 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:0a865a93:::existing_54:head# 2020-09-12T14:01:49.283 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:49.316 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:0fed513c:::existing_41:head# 2020-09-12T14:01:49.317 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:49.350 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:13b9e0b8:::existing_92:head# 2020-09-12T14:01:49.350 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:49.408 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:169157fa:::existing_81:head# 2020-09-12T14:01:49.408 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:49.442 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:17731382:::existing_58:head# 2020-09-12T14:01:49.442 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:49.475 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:18a45a39:::existing_43:head# 2020-09-12T14:01:49.475 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:49.509 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:1c160f01:::existing_66:head# 2020-09-12T14:01:49.509 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:49.542 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:1dd6c249:::existing_97:head# 2020-09-12T14:01:49.543 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:49.576 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:227e0c77:::existing_7:head# 2020-09-12T14:01:49.576 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:49.609 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:23e1b063:::existing_59:head# 2020-09-12T14:01:49.609 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:49.643 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:28566edb:::existing_78:head# 2020-09-12T14:01:49.643 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:49.718 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:2a101182:::existing_20:head# 2020-09-12T14:01:49.718 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:49.751 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:2d1519da:::existing_96:head# 2020-09-12T14:01:49.751 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:49.785 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:3078b87a:::existing_86:head# 2020-09-12T14:01:49.785 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:49.818 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:30bde01b:::existing_15:head# 2020-09-12T14:01:49.818 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:49.852 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:317a7e19:::existing_56:head# 2020-09-12T14:01:49.852 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:49.885 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:38b9c7fa:::existing_33:head# 2020-09-12T14:01:49.885 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:49.919 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:3e16d84b:::existing_48:head# 2020-09-12T14:01:49.919 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:49.974 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:45fe8203:::existing_75:head# 2020-09-12T14:01:49.974 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.002 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:4e82c9ba:::existing_50:head# 2020-09-12T14:01:50.002 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.027 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5298dcdb:::existing_57:head# 2020-09-12T14:01:50.027 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.052 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:52ebaa44:::existing_32:head# 2020-09-12T14:01:50.053 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.078 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:53588912:::existing_9:head# 2020-09-12T14:01:50.078 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.103 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5b7e2de6:::existing_84:head# 2020-09-12T14:01:50.103 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.128 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5da0255c:::existing_13:head# 2020-09-12T14:01:50.128 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.153 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5da2dab8:::existing_53:head# 2020-09-12T14:01:50.153 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.178 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5ee3514b:::existing_30:head# 2020-09-12T14:01:50.179 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.203 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:602f83fe:::foo:head# 2020-09-12T14:01:50.203 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.229 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:60426cc7:::existing_62:head# 2020-09-12T14:01:50.229 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.295 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:628918ca:::existing_79:head# 2020-09-12T14:01:50.296 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.320 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:63f8ab2d:::existing_77:head# 2020-09-12T14:01:50.320 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.345 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:67e1780f:::existing_27:head# 2020-09-12T14:01:50.346 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.370 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:68143ca1:::existing_71:head# 2020-09-12T14:01:50.371 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.396 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:6bac9509:::existing_29:head# 2020-09-12T14:01:50.396 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.420 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:6bbec798:::existing_45:head# 2020-09-12T14:01:50.421 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.446 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:6cb23a40:::existing_65:head# 2020-09-12T14:01:50.446 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.471 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7055ec5e:::existing_83:head# 2020-09-12T14:01:50.471 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.496 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:70a54e2b:::existing_87:head# 2020-09-12T14:01:50.496 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.556 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:750ace21:::existing_19:head# 2020-09-12T14:01:50.556 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.588 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:78623b1d:::existing_6:head# 2020-09-12T14:01:50.589 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.614 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:79881152:::existing_63:head# 2020-09-12T14:01:50.614 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.639 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7a80c0d5:::existing_99:head# 2020-09-12T14:01:50.639 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.664 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:81490677:::existing_76:head# 2020-09-12T14:01:50.664 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.689 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:85ec8e5d:::existing_85:head# 2020-09-12T14:01:50.689 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.714 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8e2abbf0:::existing_49:head# 2020-09-12T14:01:50.714 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.739 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8ee0fe0f:::existing_52:head# 2020-09-12T14:01:50.740 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.764 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:923c5d4d:::existing_5:head# 2020-09-12T14:01:50.764 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.789 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:92731e71:::existing_89:head# 2020-09-12T14:01:50.790 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.851 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:928205cf:::existing_82:head# 2020-09-12T14:01:50.852 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.915 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:93be2267:::existing_24:head# 2020-09-12T14:01:50.915 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.940 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:99ceda2c:::existing_44:head# 2020-09-12T14:01:50.941 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.965 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:9dd917ee:::existing_26:head# 2020-09-12T14:01:50.965 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:50.990 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a28805a6:::existing_72:head# 2020-09-12T14:01:50.991 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.016 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a294a8f6:::existing_16:head# 2020-09-12T14:01:51.016 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.041 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a2f5473a:::existing_42:head# 2020-09-12T14:01:51.041 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.066 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a504a533:::existing_90:head# 2020-09-12T14:01:51.066 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.091 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a691337e:::existing_61:head# 2020-09-12T14:01:51.091 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.191 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a6e8d7ac:::existing_28:head# 2020-09-12T14:01:51.191 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.233 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a9fde86e:::existing_18:head# 2020-09-12T14:01:51.234 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.258 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:aa309faa:::existing_68:head# 2020-09-12T14:01:51.259 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.283 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ae40b0ed:::existing_31:head# 2020-09-12T14:01:51.284 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.334 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b279b1fe:::existing_10:head# 2020-09-12T14:01:51.334 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.359 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b343e277:::existing_88:head# 2020-09-12T14:01:51.360 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.409 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b7130e90:::existing_80:head# 2020-09-12T14:01:51.409 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.443 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:bc6f7309:::existing_70:head# 2020-09-12T14:01:51.443 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.475 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:be8489b7:::existing_38:head# 2020-09-12T14:01:51.476 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.509 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:beb4e5e3:::existing_47:head# 2020-09-12T14:01:51.509 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.542 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:bebec877:::existing_74:head# 2020-09-12T14:01:51.543 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.576 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c04347d8:::existing_60:head# 2020-09-12T14:01:51.576 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.609 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c156033e:::existing_100:head# 2020-09-12T14:01:51.610 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.643 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c19f5a8e:::existing_8:head# 2020-09-12T14:01:51.643 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.710 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c3e6ebe4:::existing_17:head# 2020-09-12T14:01:51.710 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.743 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c6c5a7c2:::existing_39:head# 2020-09-12T14:01:51.743 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.776 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:cc1d0abf:::existing_95:head# 2020-09-12T14:01:51.777 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.810 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ccb3865b:::existing_73:head# 2020-09-12T14:01:51.810 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.845 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:cd84e19e:::existing_40:head# 2020-09-12T14:01:51.846 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.877 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d2cd64ce:::existing_37:head# 2020-09-12T14:01:51.877 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.910 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d41a25c9:::existing_22:head# 2020-09-12T14:01:51.911 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:51.964 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d7f088da:::existing_91:head# 2020-09-12T14:01:51.964 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:52.003 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d9cbd1d6:::existing_64:head# 2020-09-12T14:01:52.003 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:52.036 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:dcc4ff82:::existing_98:head# 2020-09-12T14:01:52.036 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:52.086 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:dd677bd3:::existing_35:head# 2020-09-12T14:01:52.087 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:52.137 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:dd6d90e7:::existing_46:head# 2020-09-12T14:01:52.137 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:52.179 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:e0e22077:::existing_21:head# 2020-09-12T14:01:52.179 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:52.220 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:e255b710:::existing_55:head# 2020-09-12T14:01:52.221 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:52.287 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:eff1d5db:::existing_14:head# 2020-09-12T14:01:52.288 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:52.329 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:f1b156bf:::existing_23:head# 2020-09-12T14:01:52.329 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:52.371 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:f5ce4199:::existing_34:head# 2020-09-12T14:01:52.371 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:52.422 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:f6809456:::existing_11:head# 2020-09-12T14:01:52.422 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:52.472 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:f699a2a9:::existing_67:head# 2020-09-12T14:01:52.472 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:52.545 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:f9baac43:::existing_25:head# 2020-09-12T14:01:52.546 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:52.597 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:fb76688e:::existing_51:head# 2020-09-12T14:01:52.598 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:52.648 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:fbb92eec:::existing_69:head# 2020-09-12T14:01:52.648 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:01:52.689 INFO:tasks.workunit.client.0.smithi027.stdout:write_pg epoch 44 info 1.0( v 44'102 lc 0'0 (29'90,44'102] local-lis/les=43/44 n=101 ec=23/23 lis/c=43/23 les/c/f=44/24/0 sis=43 pruub=13.947920054s) 2020-09-12T14:01:52.723 INFO:tasks.workunit.client.0.smithi027.stdout:Import successful 2020-09-12T14:01:53.245 INFO:tasks.workunit.client.0.smithi027.stdout:reviving divergent 1 2020-09-12T14:01:53.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:576: TEST_divergent_2: echo 'reviving divergent 1' 2020-09-12T14:01:53.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:577: TEST_divergent_2: ceph pg dump pgs 2020-09-12T14:01:53.706 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:01:53.707 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 101 4 4 0 0 48430 0 0 12 12 active+recovering+degraded 2020-09-12T14:01:46.335155+0000 44'102 44:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T13:59:19.138081+0000 0'0 2020-09-12T13:59:19.138081+0000 0 2020-09-12T14:01:53.707 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:01:53.707 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:01:53.708 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:01:53.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:578: TEST_divergent_2: activate_osd td/divergent-priors 1 2020-09-12T14:01:53.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/divergent-priors 2020-09-12T14:01:53.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T14:01:53.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=1 2020-09-12T14:01:53.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T14:01:53.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/divergent-priors/1 2020-09-12T14:01:53.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T14:01:53.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:01:53.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:01:53.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:01:53.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2020-09-12T14:01:53.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2020-09-12T14:01:53.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T14:01:53.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T14:01:53.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T14:01:53.722 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T14:01:53.722 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:01:53.722 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:01:53.722 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:01:53.722 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:01:53.722 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:01:53.723 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:01:53.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:01:53.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:01:53.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T14:01:53.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T14:01:53.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:01:53.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:01:53.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:01:53.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T14:01:53.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T14:01:53.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/divergent-priors/1 2020-09-12T14:01:53.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.1 2020-09-12T14:01:53.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 1 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:01:53.725 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/divergent-priors/1/whoami 2020-09-12T14:01:53.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 1 = 1 ']' 2020-09-12T14:01:53.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T14:01:53.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T14:01:53.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T14:01:53.726 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T14:01:53.749 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:01:53.748+0000 7f4757743ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:01:53.750 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:01:53.750+0000 7f4757743ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:01:53.752 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:01:53.751+0000 7f4757743ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:01:54.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: wait_for_osd up 1 2020-09-12T14:01:54.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:01:54.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:01:54.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:01:54.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:01:54.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:01:54.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:01:54.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:01:54.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:01:54.143 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:01:54.271 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:01:54.270+0000 7f4757743ec0 -1 Falling back to public interface 2020-09-12T14:01:54.572 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 43 up_thru 43 down_at 31 last_clean_interval [10,29) [v2:127.0.0.1:6818/40417,v1:127.0.0.1:6819/40417] [v2:127.0.0.1:6820/40417,v1:127.0.0.1:6821/40417] exists,up d92b7341-d129-4e21-85d9-c431d9f9298f 2020-09-12T14:01:54.573 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:01:54.573 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:01:54.573 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:01:54.573 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:01:54.573 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:579: TEST_divergent_2: wait_for_osd up 1 2020-09-12T14:01:54.573 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:01:54.573 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:01:54.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:01:54.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:01:54.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:01:54.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:01:54.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:01:54.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:01:55.012 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 43 up_thru 43 down_at 31 last_clean_interval [10,29) [v2:127.0.0.1:6818/40417,v1:127.0.0.1:6819/40417] [v2:127.0.0.1:6820/40417,v1:127.0.0.1:6821/40417] exists,up d92b7341-d129-4e21-85d9-c431d9f9298f 2020-09-12T14:01:55.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:01:55.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:01:55.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:01:55.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:581: TEST_divergent_2: sleep 20 2020-09-12T14:01:55.755 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:01:55.754+0000 7f4757743ec0 -1 osd.1 44 log_to_monitors {default=true} 2020-09-12T14:02:15.015 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:582: TEST_divergent_2: get_asok_path osd.1 2020-09-12T14:02:15.015 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2020-09-12T14:02:15.015 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2020-09-12T14:02:15.015 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:02:15.016 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:02:15.016 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:02:15.017 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.1.asok 2020-09-12T14:02:15.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:582: TEST_divergent_2: CEPH_ARGS= 2020-09-12T14:02:15.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:582: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.1.asok dump_ops_in_flight 2020-09-12T14:02:15.130 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:02:15.130 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [], 2020-09-12T14:02:15.130 INFO:tasks.workunit.client.0.smithi027.stdout: "num_ops": 0 2020-09-12T14:02:15.131 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:02:15.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:584: TEST_divergent_2: echo 'allowing recovery' 2020-09-12T14:02:15.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:585: TEST_divergent_2: ceph pg dump pgs 2020-09-12T14:02:15.139 INFO:tasks.workunit.client.0.smithi027.stdout:allowing recovery 2020-09-12T14:02:15.521 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:02:15.521 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 101 0 0 0 0 48430 0 0 12 12 active+clean 2020-09-12T14:01:57.863902+0000 44'102 47:148 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T13:59:19.138081+0000 0'0 2020-09-12T13:59:19.138081+0000 0 2020-09-12T14:02:15.521 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:02:15.522 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:02:15.523 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:02:15.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:587: TEST_divergent_2: for i in $osds 2020-09-12T14:02:15.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:589: TEST_divergent_2: ceph tell osd.0 debug kick_recovery_wq 0 2020-09-12T14:02:15.669 INFO:tasks.workunit.client.0.smithi027.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2020-09-12T14:02:15.678 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:587: TEST_divergent_2: for i in $osds 2020-09-12T14:02:15.678 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:589: TEST_divergent_2: ceph tell osd.1 debug kick_recovery_wq 0 2020-09-12T14:02:15.815 INFO:tasks.workunit.client.0.smithi027.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2020-09-12T14:02:15.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:587: TEST_divergent_2: for i in $osds 2020-09-12T14:02:15.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:589: TEST_divergent_2: ceph tell osd.2 debug kick_recovery_wq 0 2020-09-12T14:02:15.962 INFO:tasks.workunit.client.0.smithi027.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2020-09-12T14:02:15.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:592: TEST_divergent_2: echo 'reading divergent objects' 2020-09-12T14:02:15.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:593: TEST_divergent_2: ceph pg dump pgs 2020-09-12T14:02:15.972 INFO:tasks.workunit.client.0.smithi027.stdout:reading divergent objects 2020-09-12T14:02:16.362 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:02:16.362 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 101 0 0 0 0 48430 0 0 12 12 active+clean 2020-09-12T14:01:57.863902+0000 44'102 47:148 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T13:59:19.138081+0000 0'0 2020-09-12T13:59:19.138081+0000 0 2020-09-12T14:02:16.362 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:02:16.362 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:02:16.363 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:02:16.373 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: expr 2 + 2 2020-09-12T14:02:16.374 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: seq 1 4 2020-09-12T14:02:16.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2020-09-12T14:02:16.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:596: TEST_divergent_2: rados -p test get existing_1 td/divergent-priors/existing 2020-09-12T14:02:16.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2020-09-12T14:02:16.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:596: TEST_divergent_2: rados -p test get existing_2 td/divergent-priors/existing 2020-09-12T14:02:16.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2020-09-12T14:02:16.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:596: TEST_divergent_2: rados -p test get existing_3 td/divergent-priors/existing 2020-09-12T14:02:16.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2020-09-12T14:02:16.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:596: TEST_divergent_2: rados -p test get existing_4 td/divergent-priors/existing 2020-09-12T14:02:16.497 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:598: TEST_divergent_2: seq 1 2 2020-09-12T14:02:16.498 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:598: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2020-09-12T14:02:16.498 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:600: TEST_divergent_2: rados -p test get newobject_1 td/divergent-priors/existing 2020-09-12T14:02:16.526 INFO:tasks.workunit.client.0.smithi027.stderr:error getting test/newobject_1: (2) No such file or directory 2020-09-12T14:02:16.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:598: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2020-09-12T14:02:16.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:600: TEST_divergent_2: rados -p test get newobject_2 td/divergent-priors/existing 2020-09-12T14:02:16.557 INFO:tasks.workunit.client.0.smithi027.stderr:error getting test/newobject_2: (2) No such file or directory 2020-09-12T14:02:16.558 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:602: TEST_divergent_2: rm -f td/divergent-priors/existing 2020-09-12T14:02:16.559 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:604: TEST_divergent_2: find td/divergent-priors -name '*osd*log' 2020-09-12T14:02:16.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:604: TEST_divergent_2: grep _merge_object_divergent_entries td/divergent-priors/osd.2.log td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log 2020-09-12T14:02:16.563 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 20 _merge_object_divergent_entries: merging hoid 1:70b3bf12:::existing_4:head entries: 29'106 (29'4) delete 1:70b3bf12:::existing_4:head by client.4539.0:1 2020-09-12T13:59:50.435403+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 0, new_object: 0 2020-09-12T14:02:16.563 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 20 _merge_object_divergent_entries: keeping 29'106 (29'4) delete 1:70b3bf12:::existing_4:head by client.4539.0:1 2020-09-12T13:59:50.435403+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 0, new_object: 0 2020-09-12T14:02:16.563 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2020-09-12T14:02:16.564 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:70b3bf12:::existing_4:head prior_version: 29'4 first_divergent_update: 29'106 last_divergent_update: 29'106 2020-09-12T14:02:16.564 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: more recent entry found: 39'101 (29'4) modify 1:70b3bf12:::existing_4:head by client.4648.0:1 2020-09-12T14:00:47.485562+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0, already merged 2020-09-12T14:02:16.564 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 20 _merge_object_divergent_entries: merging hoid 1:75ac5091:::newobject_2:head entries: 29'103 (0'0) modify 1:75ac5091:::newobject_2:head by client.4531.0:1 2020-09-12T13:59:50.427979+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 2020-09-12T14:02:16.564 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 20 _merge_object_divergent_entries: keeping 29'103 (0'0) modify 1:75ac5091:::newobject_2:head by client.4531.0:1 2020-09-12T13:59:50.427979+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 2020-09-12T14:02:16.564 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2020-09-12T14:02:16.564 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version: 0'0 first_divergent_update: 29'103 last_divergent_update: 29'103 2020-09-12T14:02:16.565 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head has no more recent entries in log 2020-09-12T14:02:16.565 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version or op type indicates creation, deleting 2020-09-12T14:02:16.565 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 20 _merge_object_divergent_entries: merging hoid 1:8881e6d8:::existing_3:head entries: 29'105 (29'3) delete 1:8881e6d8:::existing_3:head by client.4537.0:1 2020-09-12T13:59:50.433433+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 0, new_object: 0 2020-09-12T14:02:16.565 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 20 _merge_object_divergent_entries: keeping 29'105 (29'3) delete 1:8881e6d8:::existing_3:head by client.4537.0:1 2020-09-12T13:59:50.433433+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 0, new_object: 0 2020-09-12T14:02:16.565 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid object_not_in_store: 1 2020-09-12T14:02:16.565 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version: 29'3 first_divergent_update: 29'105 last_divergent_update: 29'105 2020-09-12T14:02:16.565 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head has no more recent entries in log 2020-09-12T14:02:16.565 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head must be rolled back or recovered, attempting to rollback 2020-09-12T14:02:16.566 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head olog_can_rollback_to: 29'106 2020-09-12T14:02:16.566 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot rollback 29'105 (29'3) delete 1:8881e6d8:::existing_3:head by client.4537.0:1 2020-09-12T13:59:50.433433+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 0, new_object: 0 2020-09-12T14:02:16.566 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot roll back, removing and adding to missing 2020-09-12T14:02:16.566 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version 29'3 <= info.log_tail 29'90 2020-09-12T14:02:16.566 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 20 _merge_object_divergent_entries: merging hoid 1:97db2a0f:::existing_2:head entries: 29'104 (29'2) modify 1:97db2a0f:::existing_2:head by client.4534.0:1 2020-09-12T13:59:50.430653+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0 2020-09-12T14:02:16.566 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 20 _merge_object_divergent_entries: keeping 29'104 (29'2) modify 1:97db2a0f:::existing_2:head by client.4534.0:1 2020-09-12T13:59:50.430653+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0 2020-09-12T14:02:16.567 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2020-09-12T14:02:16.567 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version: 29'2 first_divergent_update: 29'104 last_divergent_update: 29'104 2020-09-12T14:02:16.567 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head has no more recent entries in log 2020-09-12T14:02:16.567 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head must be rolled back or recovered, attempting to rollback 2020-09-12T14:02:16.567 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head olog_can_rollback_to: 29'106 2020-09-12T14:02:16.567 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot rollback 29'104 (29'2) modify 1:97db2a0f:::existing_2:head by client.4534.0:1 2020-09-12T13:59:50.430653+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0 2020-09-12T14:02:16.568 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot roll back, removing and adding to missing 2020-09-12T14:02:16.568 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version 29'2 <= info.log_tail 29'90 2020-09-12T14:02:16.568 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 20 _merge_object_divergent_entries: merging hoid 1:d858c035:::existing_1:head entries: 29'102 (29'1) modify 1:d858c035:::existing_1:head by client.4527.0:1 2020-09-12T13:59:50.418133+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0 2020-09-12T14:02:16.568 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 20 _merge_object_divergent_entries: keeping 29'102 (29'1) modify 1:d858c035:::existing_1:head by client.4527.0:1 2020-09-12T13:59:50.418133+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0 2020-09-12T14:02:16.568 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2020-09-12T14:02:16.569 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version: 29'1 first_divergent_update: 29'102 last_divergent_update: 29'102 2020-09-12T14:02:16.569 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head has no more recent entries in log 2020-09-12T14:02:16.569 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head must be rolled back or recovered, attempting to rollback 2020-09-12T14:02:16.569 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head olog_can_rollback_to: 29'106 2020-09-12T14:02:16.569 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot rollback 29'102 (29'1) modify 1:d858c035:::existing_1:head by client.4527.0:1 2020-09-12T13:59:50.418133+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0 2020-09-12T14:02:16.570 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot roll back, removing and adding to missing 2020-09-12T14:02:16.570 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version 29'1 <= info.log_tail 29'90 2020-09-12T14:02:16.570 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 20 _merge_object_divergent_entries: merging hoid 1:da59f6e7:::newobject_1:head entries: 29'101 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4524.0:1 2020-09-12T13:59:50.412949+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 2020-09-12T14:02:16.570 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 20 _merge_object_divergent_entries: keeping 29'101 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4524.0:1 2020-09-12T13:59:50.412949+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 2020-09-12T14:02:16.570 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2020-09-12T14:02:16.570 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version: 0'0 first_divergent_update: 29'101 last_divergent_update: 29'101 2020-09-12T14:02:16.570 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head has no more recent entries in log 2020-09-12T14:02:16.571 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:01:32.000+0000 7f0adf52b700 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version or op type indicates creation, deleting 2020-09-12T14:02:16.571 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:606: TEST_divergent_2: find td/divergent-priors -name '*osd*log' 2020-09-12T14:02:16.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:606: TEST_divergent_2: grep -q '_merge_object_divergent_entries: more recent entry found:' td/divergent-priors/osd.2.log td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log 2020-09-12T14:02:16.572 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:612: TEST_divergent_2: find td/divergent-priors -name '*osd*log' 2020-09-12T14:02:16.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:612: TEST_divergent_2: grep -q '_merge_object_divergent_entries.*prior_version or op type indicates creation' td/divergent-priors/osd.2.log td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log 2020-09-12T14:02:16.573 INFO:tasks.workunit.client.0.smithi027.stdout:success 2020-09-12T14:02:16.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:617: TEST_divergent_2: echo success 2020-09-12T14:02:16.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:619: TEST_divergent_2: rm td/divergent-priors/td/divergent-priors/exp.20815.out 2020-09-12T14:02:16.574 INFO:tasks.workunit.client.0.smithi027.stderr:rm: cannot remove 'td/divergent-priors/td/divergent-priors/exp.20815.out': No such file or directory 2020-09-12T14:02:16.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:621: TEST_divergent_2: delete_pool test 2020-09-12T14:02:16.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: delete_pool: local poolname=test 2020-09-12T14:02:16.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2020-09-12T14:02:17.093 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test' does not exist 2020-09-12T14:02:17.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:622: TEST_divergent_2: kill_daemons td/divergent-priors 2020-09-12T14:02:17.103 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:02:17.103 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:02:17.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:02:17.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:02:17.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:02:17.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:02:17.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2020-09-12T14:02:17.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2020-09-12T14:02:17.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:02:17.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2020-09-12T14:02:17.212 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:02:17.213 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:02:17.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:02:17.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:02:17.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:02:17.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:02:17.217 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:02:17.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:02:17.218 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:02:17.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:02:17.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:02:17.219 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:02:17.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:02:17.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:02:17.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:02:17.222 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:02:17.222 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:02:17.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:02:17.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2020-09-12T14:02:17.234 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:02:17.235 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:02:17.235 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:02:17.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.20815 2020-09-12T14:02:17.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:02:17.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:02:17.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2020-09-12T14:02:17.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2020-09-12T14:02:17.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/divergent-priors 2020-09-12T14:02:17.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/divergent-priors 2020-09-12T14:02:17.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2020-09-12T14:02:17.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:02:17.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2020-09-12T14:02:17.238 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:02:17.238 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:02:17.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:02:17.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:02:17.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:02:17.240 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:02:17.240 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:02:17.240 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:02:17.240 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:02:17.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:02:17.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:02:17.241 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:02:17.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:02:17.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:02:17.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:02:17.242 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:02:17.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:02:17.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:02:17.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2020-09-12T14:02:17.245 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:02:17.245 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:02:17.245 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:02:17.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.20815 2020-09-12T14:02:17.246 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:02:17.246 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:02:17.246 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/divergent-priors 2020-09-12T14:02:17.247 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2020-09-12T14:02:17.247 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:02:17.247 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:02:17.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.20815 2020-09-12T14:02:17.249 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2020-09-12T14:02:17.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2020-09-12T14:02:17.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2020-09-12T14:02:17.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2020-09-12T14:02:17.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent_3 td/divergent-priors 2020-09-12T14:02:17.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:628: TEST_divergent_3: local dir=td/divergent-priors 2020-09-12T14:02:17.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:631: TEST_divergent_3: local dummyfile=/etc/fstab 2020-09-12T14:02:17.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:632: TEST_divergent_3: local dummyfile2=/etc/resolv.conf 2020-09-12T14:02:17.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:634: TEST_divergent_3: local num_osds=3 2020-09-12T14:02:17.251 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:635: TEST_divergent_3: expr 3 - 1 2020-09-12T14:02:17.252 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:635: TEST_divergent_3: seq 0 2 2020-09-12T14:02:17.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:635: TEST_divergent_3: local 'osds=0 2020-09-12T14:02:17.252 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:02:17.252 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T14:02:17.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:636: TEST_divergent_3: run_mon td/divergent-priors a 2020-09-12T14:02:17.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/divergent-priors 2020-09-12T14:02:17.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2020-09-12T14:02:17.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2020-09-12T14:02:17.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2020-09-12T14:02:17.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/divergent-priors/a 2020-09-12T14:02:17.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2020-09-12T14:02:17.552 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2020-09-12T14:02:17.552 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:02:17.552 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:02:17.552 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:02:17.553 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:02:17.553 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:02:17.553 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:02:17.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 2020-09-12T14:02:17.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: cat 2020-09-12T14:02:17.795 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a fsid 2020-09-12T14:02:17.796 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:02:17.796 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:02:17.796 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=fsid 2020-09-12T14:02:17.797 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .fsid 2020-09-12T14:02:17.797 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:02:17.797 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:02:17.797 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:02:17.797 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:02:17.797 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:02:17.798 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:02:17.798 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-mon.a.asok 2020-09-12T14:02:17.798 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:02:17.798 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.20815/ceph-mon.a.asok config get fsid 2020-09-12T14:02:17.904 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a mon_host 2020-09-12T14:02:17.904 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:02:17.904 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:02:17.904 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=mon_host 2020-09-12T14:02:17.905 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .mon_host 2020-09-12T14:02:17.905 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:02:17.905 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:02:17.905 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:02:17.906 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:02:17.907 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:02:17.907 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:02:17.907 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-mon.a.asok 2020-09-12T14:02:17.907 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:02:17.907 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.20815/ceph-mon.a.asok config get mon_host 2020-09-12T14:02:18.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:637: TEST_divergent_3: run_mgr td/divergent-priors x 2020-09-12T14:02:18.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: local dir=td/divergent-priors 2020-09-12T14:02:18.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: shift 2020-09-12T14:02:18.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: local id=x 2020-09-12T14:02:18.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: shift 2020-09-12T14:02:18.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: local data=td/divergent-priors/x 2020-09-12T14:02:18.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:558: run_mgr: ceph config set mgr mgr/devicehealth/enable_monitoring off --force 2020-09-12T14:02:18.916 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: get_asok_path 2020-09-12T14:02:18.917 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:02:18.917 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:02:18.917 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:02:18.917 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:02:18.917 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:02:18.917 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:02:18.918 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:02:18.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:02:18.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:638: TEST_divergent_3: for i in $osds 2020-09-12T14:02:18.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:640: TEST_divergent_3: run_osd td/divergent-priors 0 2020-09-12T14:02:18.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/divergent-priors 2020-09-12T14:02:18.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:02:18.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=0 2020-09-12T14:02:18.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:02:18.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/divergent-priors/0 2020-09-12T14:02:18.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T14:02:18.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:02:18.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:02:18.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:02:18.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2020-09-12T14:02:18.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2020-09-12T14:02:18.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:02:18.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:02:18.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T14:02:18.948 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:02:18.948 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:02:18.948 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:02:18.948 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:02:18.948 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:02:18.948 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:02:18.949 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:02:18.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:02:18.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:02:18.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:02:18.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:02:18.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T14:02:18.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T14:02:18.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:02:18.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:02:18.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:02:18.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:02:18.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:02:18.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/divergent-priors/0 2020-09-12T14:02:18.952 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:02:18.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=9d8d2ee9-0710-4a4e-a0ba-cec45cbf29a2 2020-09-12T14:02:18.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd0 9d8d2ee9-0710-4a4e-a0ba-cec45cbf29a2' 2020-09-12T14:02:18.952 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:02:18.953 INFO:tasks.workunit.client.0.smithi027.stdout:add osd0 9d8d2ee9-0710-4a4e-a0ba-cec45cbf29a2 2020-09-12T14:02:18.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDq1FxfnauhORAAkZ0HgAbG1PeB8aGTym7IoQ== 2020-09-12T14:02:18.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDq1FxfnauhORAAkZ0HgAbG1PeB8aGTym7IoQ=="}' 2020-09-12T14:02:18.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 9d8d2ee9-0710-4a4e-a0ba-cec45cbf29a2 -i td/divergent-priors/0/new.json 2020-09-12T14:02:19.330 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:02:19.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/divergent-priors/0/new.json 2020-09-12T14:02:19.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 0 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDq1FxfnauhORAAkZ0HgAbG1PeB8aGTym7IoQ== --osd-uuid 9d8d2ee9-0710-4a4e-a0ba-cec45cbf29a2 2020-09-12T14:02:19.372 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:19.371+0000 7f130cb03ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:02:19.374 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:19.373+0000 7f130cb03ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:02:19.375 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:19.375+0000 7f130cb03ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:02:19.376 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:19.375+0000 7f130cb03ec0 -1 bluestore(td/divergent-priors/0/block) _read_bdev_label failed to open td/divergent-priors/0/block: (2) No such file or directory 2020-09-12T14:02:19.376 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:19.375+0000 7f130cb03ec0 -1 bluestore(td/divergent-priors/0/block) _read_bdev_label failed to open td/divergent-priors/0/block: (2) No such file or directory 2020-09-12T14:02:19.376 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:19.375+0000 7f130cb03ec0 -1 bluestore(td/divergent-priors/0/block) _read_bdev_label failed to open td/divergent-priors/0/block: (2) No such file or directory 2020-09-12T14:02:19.428 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:19.427+0000 7f130cb03ec0 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2020-09-12T14:02:23.672 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/divergent-priors/0/keyring 2020-09-12T14:02:23.673 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:02:23.674 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd0 key to auth repository 2020-09-12T14:02:23.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd0 key to auth repository 2020-09-12T14:02:23.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:02:24.231 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.0 2020-09-12T14:02:24.231 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.0 2020-09-12T14:02:24.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 0 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:02:24.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:02:24.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:02:24.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:02:24.260 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:24.259+0000 7f17abf97ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:02:24.269 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:24.268+0000 7f17abf97ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:02:24.270 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:24.269+0000 7f17abf97ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:02:24.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 0 2020-09-12T14:02:24.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:02:24.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T14:02:24.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:02:24.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:02:24.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:02:24.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:02:24.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:02:24.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:02:24.658 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:02:24.787 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:24.787+0000 7f17abf97ec0 -1 Falling back to public interface 2020-09-12T14:02:25.066 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:02:25.612 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:25.611+0000 7f17abf97ec0 -1 osd.0 0 log_to_monitors {default=true} 2020-09-12T14:02:26.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:02:26.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:02:26.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:02:26.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:02:26.068 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:02:26.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:02:26.489 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/43132,v1:127.0.0.1:6803/43132] [v2:127.0.0.1:6804/43132,v1:127.0.0.1:6805/43132] exists,up 9d8d2ee9-0710-4a4e-a0ba-cec45cbf29a2 2020-09-12T14:02:26.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:02:26.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:02:26.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:02:26.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:638: TEST_divergent_3: for i in $osds 2020-09-12T14:02:26.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:640: TEST_divergent_3: run_osd td/divergent-priors 1 2020-09-12T14:02:26.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/divergent-priors 2020-09-12T14:02:26.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:02:26.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=1 2020-09-12T14:02:26.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:02:26.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/divergent-priors/1 2020-09-12T14:02:26.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T14:02:26.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:02:26.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:02:26.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:02:26.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2020-09-12T14:02:26.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2020-09-12T14:02:26.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:02:26.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:02:26.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T14:02:26.492 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:02:26.492 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:02:26.492 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:02:26.492 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:02:26.492 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:02:26.493 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:02:26.493 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:02:26.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:02:26.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:02:26.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:02:26.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:02:26.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T14:02:26.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T14:02:26.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:02:26.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:02:26.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:02:26.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:02:26.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:02:26.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/divergent-priors/1 2020-09-12T14:02:26.495 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:02:26.496 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=06975646-fe2b-4ca5-82e0-a2d0e01e06a8 2020-09-12T14:02:26.496 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd1 06975646-fe2b-4ca5-82e0-a2d0e01e06a8' 2020-09-12T14:02:26.496 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:02:26.496 INFO:tasks.workunit.client.0.smithi027.stdout:add osd1 06975646-fe2b-4ca5-82e0-a2d0e01e06a8 2020-09-12T14:02:26.510 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDy1FxfoatbHhAAuHiXNDodsCZgUXZfsTa2MQ== 2020-09-12T14:02:26.510 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDy1FxfoatbHhAAuHiXNDodsCZgUXZfsTa2MQ=="}' 2020-09-12T14:02:26.510 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 06975646-fe2b-4ca5-82e0-a2d0e01e06a8 -i td/divergent-priors/1/new.json 2020-09-12T14:02:27.035 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:02:27.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/divergent-priors/1/new.json 2020-09-12T14:02:27.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 1 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDy1FxfoatbHhAAuHiXNDodsCZgUXZfsTa2MQ== --osd-uuid 06975646-fe2b-4ca5-82e0-a2d0e01e06a8 2020-09-12T14:02:27.078 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:27.077+0000 7f8fe754cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:02:27.080 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:27.079+0000 7f8fe754cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:02:27.081 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:27.080+0000 7f8fe754cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:02:27.082 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:27.081+0000 7f8fe754cec0 -1 bluestore(td/divergent-priors/1/block) _read_bdev_label failed to open td/divergent-priors/1/block: (2) No such file or directory 2020-09-12T14:02:27.082 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:27.081+0000 7f8fe754cec0 -1 bluestore(td/divergent-priors/1/block) _read_bdev_label failed to open td/divergent-priors/1/block: (2) No such file or directory 2020-09-12T14:02:27.082 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:27.081+0000 7f8fe754cec0 -1 bluestore(td/divergent-priors/1/block) _read_bdev_label failed to open td/divergent-priors/1/block: (2) No such file or directory 2020-09-12T14:02:27.128 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:27.127+0000 7f8fe754cec0 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2020-09-12T14:02:31.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/divergent-priors/1/keyring 2020-09-12T14:02:31.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:02:31.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd1 key to auth repository 2020-09-12T14:02:31.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:02:31.721 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd1 key to auth repository 2020-09-12T14:02:32.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.1 2020-09-12T14:02:32.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 1 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:02:32.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:02:32.272 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T14:02:32.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:02:32.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:02:32.302 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:32.302+0000 7f521ea88ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:02:32.304 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:32.304+0000 7f521ea88ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:02:32.305 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:32.305+0000 7f521ea88ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:02:32.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 1 2020-09-12T14:02:32.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:02:32.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:02:32.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:02:32.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:02:32.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:02:32.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:02:32.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:02:32.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:02:32.682 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:02:32.823 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:32.823+0000 7f521ea88ec0 -1 Falling back to public interface 2020-09-12T14:02:33.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:02:33.601 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:33.600+0000 7f521ea88ec0 -1 osd.1 0 log_to_monitors {default=true} 2020-09-12T14:02:34.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:02:34.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:02:34.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:02:34.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:02:34.126 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:02:34.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:02:34.562 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/44126,v1:127.0.0.1:6811/44126] [v2:127.0.0.1:6812/44126,v1:127.0.0.1:6813/44126] exists,up 06975646-fe2b-4ca5-82e0-a2d0e01e06a8 2020-09-12T14:02:34.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:02:34.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:02:34.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:02:34.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:638: TEST_divergent_3: for i in $osds 2020-09-12T14:02:34.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:640: TEST_divergent_3: run_osd td/divergent-priors 2 2020-09-12T14:02:34.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/divergent-priors 2020-09-12T14:02:34.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:02:34.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=2 2020-09-12T14:02:34.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:02:34.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/divergent-priors/2 2020-09-12T14:02:34.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T14:02:34.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:02:34.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:02:34.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:02:34.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2020-09-12T14:02:34.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2020-09-12T14:02:34.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:02:34.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:02:34.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T14:02:34.565 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:02:34.565 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:02:34.566 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:02:34.566 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:02:34.566 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:02:34.566 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:02:34.566 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:02:34.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:02:34.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:02:34.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:02:34.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:02:34.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T14:02:34.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T14:02:34.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:02:34.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:02:34.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:02:34.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:02:34.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:02:34.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/divergent-priors/2 2020-09-12T14:02:34.570 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:02:34.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=ed2b43d9-2a50-45c4-b6a7-8f00f86304e8 2020-09-12T14:02:34.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd2 ed2b43d9-2a50-45c4-b6a7-8f00f86304e8' 2020-09-12T14:02:34.571 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:02:34.571 INFO:tasks.workunit.client.0.smithi027.stdout:add osd2 ed2b43d9-2a50-45c4-b6a7-8f00f86304e8 2020-09-12T14:02:34.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQD61FxfRLfLIhAAKZ07TON50+Y6IG9mMRqgLQ== 2020-09-12T14:02:34.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQD61FxfRLfLIhAAKZ07TON50+Y6IG9mMRqgLQ=="}' 2020-09-12T14:02:34.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new ed2b43d9-2a50-45c4-b6a7-8f00f86304e8 -i td/divergent-priors/2/new.json 2020-09-12T14:02:35.134 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:02:35.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/divergent-priors/2/new.json 2020-09-12T14:02:35.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 2 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQD61FxfRLfLIhAAKZ07TON50+Y6IG9mMRqgLQ== --osd-uuid ed2b43d9-2a50-45c4-b6a7-8f00f86304e8 2020-09-12T14:02:35.177 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:35.176+0000 7fae6913dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:02:35.178 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:35.177+0000 7fae6913dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:02:35.180 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:35.179+0000 7fae6913dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:02:35.180 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:35.180+0000 7fae6913dec0 -1 bluestore(td/divergent-priors/2/block) _read_bdev_label failed to open td/divergent-priors/2/block: (2) No such file or directory 2020-09-12T14:02:35.181 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:35.180+0000 7fae6913dec0 -1 bluestore(td/divergent-priors/2/block) _read_bdev_label failed to open td/divergent-priors/2/block: (2) No such file or directory 2020-09-12T14:02:35.181 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:35.180+0000 7fae6913dec0 -1 bluestore(td/divergent-priors/2/block) _read_bdev_label failed to open td/divergent-priors/2/block: (2) No such file or directory 2020-09-12T14:02:35.252 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:35.252+0000 7fae6913dec0 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2020-09-12T14:02:39.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/divergent-priors/2/keyring 2020-09-12T14:02:39.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:02:39.845 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd2 key to auth repository 2020-09-12T14:02:39.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd2 key to auth repository 2020-09-12T14:02:39.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:02:40.404 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.2 2020-09-12T14:02:40.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.2 2020-09-12T14:02:40.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 2 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:02:40.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:02:40.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:02:40.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:02:40.434 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:40.431+0000 7fbd2c97bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:02:40.435 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:40.433+0000 7fbd2c97bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:02:40.437 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:40.434+0000 7fbd2c97bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:02:40.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 2 2020-09-12T14:02:40.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:02:40.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T14:02:40.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:02:40.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:02:40.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:02:40.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:02:40.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:02:40.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:02:40.836 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:02:40.959 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:40.957+0000 7fbd2c97bec0 -1 Falling back to public interface 2020-09-12T14:02:41.262 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:02:41.745 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:02:41.743+0000 7fbd2c97bec0 -1 osd.2 0 log_to_monitors {default=true} 2020-09-12T14:02:42.264 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:02:42.264 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:02:42.264 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:02:42.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:02:42.265 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:02:42.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:02:42.736 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/45120,v1:127.0.0.1:6819/45120] [v2:127.0.0.1:6820/45120,v1:127.0.0.1:6821/45120] exists,up ed2b43d9-2a50-45c4-b6a7-8f00f86304e8 2020-09-12T14:02:42.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:02:42.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:02:42.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:02:42.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:643: TEST_divergent_3: ceph osd set noout 2020-09-12T14:02:43.513 INFO:tasks.workunit.client.0.smithi027.stderr:noout is set 2020-09-12T14:02:43.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:644: TEST_divergent_3: ceph osd set noin 2020-09-12T14:02:44.383 INFO:tasks.workunit.client.0.smithi027.stderr:noin is set 2020-09-12T14:02:44.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:645: TEST_divergent_3: ceph osd set nodown 2020-09-12T14:02:45.193 INFO:tasks.workunit.client.0.smithi027.stderr:nodown is set 2020-09-12T14:02:45.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:646: TEST_divergent_3: create_pool test 1 1 2020-09-12T14:02:45.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test 1 1 2020-09-12T14:02:45.860 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test' already exists 2020-09-12T14:02:45.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:02:46.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:647: TEST_divergent_3: ceph osd pool set test size 3 2020-09-12T14:02:47.628 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 1 size to 3 2020-09-12T14:02:47.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:648: TEST_divergent_3: ceph osd pool set test min_size 2 2020-09-12T14:02:48.388 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 1 min_size to 2 2020-09-12T14:02:48.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:651: TEST_divergent_3: ceph osd pool set test pg_autoscale_mode on 2020-09-12T14:02:49.058 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 1 pg_autoscale_mode to on 2020-09-12T14:02:49.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:653: TEST_divergent_3: flush_pg_stats 2020-09-12T14:02:49.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:02:49.068 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:02:49.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:02:49.483 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:02:49.484 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T14:02:49.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:02:49.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:02:49.484 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:02:49.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836487 2020-09-12T14:02:49.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836487 2020-09-12T14:02:49.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836487' 2020-09-12T14:02:49.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:02:49.631 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:02:49.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672965 2020-09-12T14:02:49.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672965 2020-09-12T14:02:49.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836487 1-42949672965' 2020-09-12T14:02:49.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:02:49.779 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:02:49.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509444 2020-09-12T14:02:49.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509444 2020-09-12T14:02:49.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836487 1-42949672965 2-64424509444' 2020-09-12T14:02:49.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:02:49.925 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836487 2020-09-12T14:02:49.926 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:02:49.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:02:49.926 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836487 2020-09-12T14:02:49.926 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:02:49.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836487 2020-09-12T14:02:49.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2020-09-12T14:02:49.927 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836487 2020-09-12T14:02:49.928 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:02:50.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836485 -lt 21474836487 2020-09-12T14:02:50.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:02:51.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:02:51.335 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:02:51.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836487 -lt 21474836487 2020-09-12T14:02:51.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:02:51.760 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672965 2020-09-12T14:02:51.760 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:02:51.762 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:02:51.762 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672965 2020-09-12T14:02:51.762 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:02:51.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672965 2020-09-12T14:02:51.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672965' 2020-09-12T14:02:51.763 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:02:51.764 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672965 2020-09-12T14:02:52.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672966 -lt 42949672965 2020-09-12T14:02:52.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:02:52.176 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509444 2020-09-12T14:02:52.176 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:02:52.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:02:52.177 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509444 2020-09-12T14:02:52.177 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:02:52.178 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509444 2020-09-12T14:02:52.178 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509444' 2020-09-12T14:02:52.178 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509444 2020-09-12T14:02:52.179 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:02:52.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509444 -lt 64424509444 2020-09-12T14:02:52.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:654: TEST_divergent_3: wait_for_clean 2020-09-12T14:02:52.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:02:52.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:02:52.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:02:52.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:02:52.596 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 90 .1 2020-09-12T14:02:52.596 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:02:52.596 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:02:52.597 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:02:52.597 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:02:52.597 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:02:52.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:02:52.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:02:52.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:02:52.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:02:52.691 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:02:53.122 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:02:53.122 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:02:53.122 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T14:02:53.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:02:53.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:02:53.123 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:02:53.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836490 2020-09-12T14:02:53.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836490 2020-09-12T14:02:53.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836490' 2020-09-12T14:02:53.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:02:53.270 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:02:53.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672968 2020-09-12T14:02:53.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672968 2020-09-12T14:02:53.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836490 1-42949672968' 2020-09-12T14:02:53.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:02:53.416 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:02:53.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509447 2020-09-12T14:02:53.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509447 2020-09-12T14:02:53.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836490 1-42949672968 2-64424509447' 2020-09-12T14:02:53.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:02:53.565 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836490 2020-09-12T14:02:53.565 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:02:53.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:02:53.566 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836490 2020-09-12T14:02:53.566 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:02:53.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836490 2020-09-12T14:02:53.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2020-09-12T14:02:53.567 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:02:53.567 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836490 2020-09-12T14:02:53.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836488 -lt 21474836490 2020-09-12T14:02:53.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:02:54.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:02:54.972 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:02:55.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836490 -lt 21474836490 2020-09-12T14:02:55.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:02:55.398 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672968 2020-09-12T14:02:55.398 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:02:55.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:02:55.398 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672968 2020-09-12T14:02:55.399 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:02:55.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672968 2020-09-12T14:02:55.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672968' 2020-09-12T14:02:55.400 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:02:55.400 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672968 2020-09-12T14:02:55.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672968 -lt 42949672968 2020-09-12T14:02:55.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:02:55.805 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509447 2020-09-12T14:02:55.805 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:02:55.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:02:55.806 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509447 2020-09-12T14:02:55.807 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:02:55.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509447 2020-09-12T14:02:55.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509447' 2020-09-12T14:02:55.808 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509447 2020-09-12T14:02:55.808 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:02:56.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509447 -lt 64424509447 2020-09-12T14:02:56.214 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:02:56.214 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:02:56.214 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:02:56.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 1 == 0 2020-09-12T14:02:56.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:02:56.752 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:02:56.752 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:02:56.752 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:02:56.752 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:02:56.753 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:02:56.753 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:02:57.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:02:57.149 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:02:57.149 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:02:57.149 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:02:57.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 1 2020-09-12T14:02:57.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T14:02:57.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T14:02:57.684 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:657: TEST_divergent_3: ceph pg dump pgs --format=json 2020-09-12T14:02:57.685 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:657: TEST_divergent_3: jq '.pg_stats[0].up_primary' 2020-09-12T14:02:58.070 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:02:58.079 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:657: TEST_divergent_3: local divergent=1 2020-09-12T14:02:58.079 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:658: TEST_divergent_3: echo 'primary and soon to be divergent is 1' 2020-09-12T14:02:58.079 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:659: TEST_divergent_3: ceph pg dump pgs 2020-09-12T14:02:58.080 INFO:tasks.workunit.client.0.smithi027.stdout:primary and soon to be divergent is 1 2020-09-12T14:02:58.461 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:02:58.462 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 0 0 0 0 0 0 0 0 0 0 active+clean 2020-09-12T14:02:46.938591+0000 0'0 26:11 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:02:58.462 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:02:58.462 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:02:58.463 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:02:58.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:660: TEST_divergent_3: local non_divergent= 2020-09-12T14:02:58.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:661: TEST_divergent_3: for i in $osds 2020-09-12T14:02:58.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:663: TEST_divergent_3: '[' 0 = 1 ']' 2020-09-12T14:02:58.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:666: TEST_divergent_3: non_divergent=' 0' 2020-09-12T14:02:58.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:661: TEST_divergent_3: for i in $osds 2020-09-12T14:02:58.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:663: TEST_divergent_3: '[' 1 = 1 ']' 2020-09-12T14:02:58.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:664: TEST_divergent_3: continue 2020-09-12T14:02:58.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:661: TEST_divergent_3: for i in $osds 2020-09-12T14:02:58.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:663: TEST_divergent_3: '[' 2 = 1 ']' 2020-09-12T14:02:58.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:666: TEST_divergent_3: non_divergent=' 0 2' 2020-09-12T14:02:58.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:669: TEST_divergent_3: echo 'writing initial objects' 2020-09-12T14:02:58.475 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: seq 1 100 2020-09-12T14:02:58.475 INFO:tasks.workunit.client.0.smithi027.stdout:writing initial objects 2020-09-12T14:02:58.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:02:58.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_1 /etc/fstab 2020-09-12T14:02:58.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:02:58.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_2 /etc/fstab 2020-09-12T14:02:58.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:02:58.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_3 /etc/fstab 2020-09-12T14:02:58.894 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:02:58.894 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_4 /etc/fstab 2020-09-12T14:02:59.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:02:59.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_5 /etc/fstab 2020-09-12T14:02:59.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:02:59.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_6 /etc/fstab 2020-09-12T14:02:59.354 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:02:59.355 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_7 /etc/fstab 2020-09-12T14:02:59.480 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:02:59.480 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_8 /etc/fstab 2020-09-12T14:02:59.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:02:59.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_9 /etc/fstab 2020-09-12T14:02:59.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:02:59.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_10 /etc/fstab 2020-09-12T14:02:59.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:02:59.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_11 /etc/fstab 2020-09-12T14:03:00.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:00.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_12 /etc/fstab 2020-09-12T14:03:00.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:00.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_13 /etc/fstab 2020-09-12T14:03:00.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:00.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_14 /etc/fstab 2020-09-12T14:03:00.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:00.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_15 /etc/fstab 2020-09-12T14:03:00.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:00.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_16 /etc/fstab 2020-09-12T14:03:00.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:00.771 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_17 /etc/fstab 2020-09-12T14:03:00.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:00.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_18 /etc/fstab 2020-09-12T14:03:01.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:01.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_19 /etc/fstab 2020-09-12T14:03:01.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:01.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_20 /etc/fstab 2020-09-12T14:03:01.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:01.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_21 /etc/fstab 2020-09-12T14:03:01.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:01.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_22 /etc/fstab 2020-09-12T14:03:01.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:01.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_23 /etc/fstab 2020-09-12T14:03:01.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:01.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_24 /etc/fstab 2020-09-12T14:03:01.894 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:01.894 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_25 /etc/fstab 2020-09-12T14:03:02.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:02.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_26 /etc/fstab 2020-09-12T14:03:02.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:02.423 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_27 /etc/fstab 2020-09-12T14:03:02.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:02.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_28 /etc/fstab 2020-09-12T14:03:02.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:02.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_29 /etc/fstab 2020-09-12T14:03:02.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:02.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_30 /etc/fstab 2020-09-12T14:03:02.782 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:02.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_31 /etc/fstab 2020-09-12T14:03:02.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:02.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_32 /etc/fstab 2020-09-12T14:03:03.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:03.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_33 /etc/fstab 2020-09-12T14:03:03.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:03.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_34 /etc/fstab 2020-09-12T14:03:03.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:03.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_35 /etc/fstab 2020-09-12T14:03:03.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:03.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_36 /etc/fstab 2020-09-12T14:03:03.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:03.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_37 /etc/fstab 2020-09-12T14:03:03.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:03.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_38 /etc/fstab 2020-09-12T14:03:03.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:03.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_39 /etc/fstab 2020-09-12T14:03:04.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:04.066 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_40 /etc/fstab 2020-09-12T14:03:04.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:04.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_41 /etc/fstab 2020-09-12T14:03:04.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:04.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_42 /etc/fstab 2020-09-12T14:03:04.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:04.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_43 /etc/fstab 2020-09-12T14:03:04.661 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:04.662 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_44 /etc/fstab 2020-09-12T14:03:04.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:04.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_45 /etc/fstab 2020-09-12T14:03:04.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:04.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_46 /etc/fstab 2020-09-12T14:03:05.122 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:05.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_47 /etc/fstab 2020-09-12T14:03:05.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:05.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_48 /etc/fstab 2020-09-12T14:03:05.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:05.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_49 /etc/fstab 2020-09-12T14:03:05.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:05.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_50 /etc/fstab 2020-09-12T14:03:05.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:05.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_51 /etc/fstab 2020-09-12T14:03:05.859 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:05.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_52 /etc/fstab 2020-09-12T14:03:05.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:05.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_53 /etc/fstab 2020-09-12T14:03:06.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:06.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_54 /etc/fstab 2020-09-12T14:03:06.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:06.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_55 /etc/fstab 2020-09-12T14:03:06.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:06.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_56 /etc/fstab 2020-09-12T14:03:06.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:06.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_57 /etc/fstab 2020-09-12T14:03:06.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:06.848 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_58 /etc/fstab 2020-09-12T14:03:07.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:07.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_59 /etc/fstab 2020-09-12T14:03:07.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:07.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_60 /etc/fstab 2020-09-12T14:03:07.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:07.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_61 /etc/fstab 2020-09-12T14:03:07.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:07.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_62 /etc/fstab 2020-09-12T14:03:07.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:07.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_63 /etc/fstab 2020-09-12T14:03:07.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:07.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_64 /etc/fstab 2020-09-12T14:03:07.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:07.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_65 /etc/fstab 2020-09-12T14:03:07.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:07.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_66 /etc/fstab 2020-09-12T14:03:08.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:08.172 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_67 /etc/fstab 2020-09-12T14:03:08.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:08.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_68 /etc/fstab 2020-09-12T14:03:08.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:08.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_69 /etc/fstab 2020-09-12T14:03:08.590 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:08.590 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_70 /etc/fstab 2020-09-12T14:03:08.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:08.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_71 /etc/fstab 2020-09-12T14:03:08.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:08.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_72 /etc/fstab 2020-09-12T14:03:09.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:09.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_73 /etc/fstab 2020-09-12T14:03:09.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:09.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_74 /etc/fstab 2020-09-12T14:03:09.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:09.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_75 /etc/fstab 2020-09-12T14:03:09.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:09.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_76 /etc/fstab 2020-09-12T14:03:09.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:09.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_77 /etc/fstab 2020-09-12T14:03:09.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:09.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_78 /etc/fstab 2020-09-12T14:03:09.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:09.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_79 /etc/fstab 2020-09-12T14:03:09.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:09.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_80 /etc/fstab 2020-09-12T14:03:10.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:10.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_81 /etc/fstab 2020-09-12T14:03:10.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:10.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_82 /etc/fstab 2020-09-12T14:03:10.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:10.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_83 /etc/fstab 2020-09-12T14:03:10.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:10.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_84 /etc/fstab 2020-09-12T14:03:10.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:10.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_85 /etc/fstab 2020-09-12T14:03:10.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:10.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_86 /etc/fstab 2020-09-12T14:03:10.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:10.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_87 /etc/fstab 2020-09-12T14:03:11.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:11.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_88 /etc/fstab 2020-09-12T14:03:11.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:11.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_89 /etc/fstab 2020-09-12T14:03:11.423 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:11.423 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_90 /etc/fstab 2020-09-12T14:03:11.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:11.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_91 /etc/fstab 2020-09-12T14:03:11.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:11.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_92 /etc/fstab 2020-09-12T14:03:11.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:11.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_93 /etc/fstab 2020-09-12T14:03:11.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:11.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_94 /etc/fstab 2020-09-12T14:03:12.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:12.119 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_95 /etc/fstab 2020-09-12T14:03:12.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:12.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_96 /etc/fstab 2020-09-12T14:03:12.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:12.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_97 /etc/fstab 2020-09-12T14:03:12.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:12.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_98 /etc/fstab 2020-09-12T14:03:12.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:12.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_99 /etc/fstab 2020-09-12T14:03:12.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: for i in $(seq 1 $testobjects) 2020-09-12T14:03:12.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:673: TEST_divergent_3: rados -p test put existing_100 /etc/fstab 2020-09-12T14:03:12.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:676: TEST_divergent_3: WAIT_FOR_CLEAN_TIMEOUT=20 2020-09-12T14:03:12.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:676: TEST_divergent_3: wait_for_clean 2020-09-12T14:03:12.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:03:12.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:03:12.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:03:12.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:03:12.977 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 20 .1 2020-09-12T14:03:12.977 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:03:12.978 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:03:12.978 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:03:12.978 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:03:12.978 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:03:13.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:03:13.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:03:13.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:03:13.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:03:13.041 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:03:13.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:03:13.452 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:03:13.452 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T14:03:13.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:03:13.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:03:13.453 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:03:13.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836496 2020-09-12T14:03:13.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836496 2020-09-12T14:03:13.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836496' 2020-09-12T14:03:13.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:03:13.600 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:03:13.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672974 2020-09-12T14:03:13.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672974 2020-09-12T14:03:13.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836496 1-42949672974' 2020-09-12T14:03:13.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:03:13.748 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:03:13.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509453 2020-09-12T14:03:13.894 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509453 2020-09-12T14:03:13.894 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836496 1-42949672974 2-64424509453' 2020-09-12T14:03:13.894 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:03:13.895 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836496 2020-09-12T14:03:13.895 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:03:13.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:03:13.895 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836496 2020-09-12T14:03:13.896 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:03:13.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836496 2020-09-12T14:03:13.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836496' 2020-09-12T14:03:13.897 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836496 2020-09-12T14:03:13.897 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:03:14.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836494 -lt 21474836496 2020-09-12T14:03:14.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:03:15.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:03:15.308 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:03:15.730 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836496 -lt 21474836496 2020-09-12T14:03:15.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:03:15.731 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672974 2020-09-12T14:03:15.731 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:03:15.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:03:15.732 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672974 2020-09-12T14:03:15.732 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:03:15.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672974 2020-09-12T14:03:15.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672974' 2020-09-12T14:03:15.733 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672974 2020-09-12T14:03:15.734 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:03:16.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672974 -lt 42949672974 2020-09-12T14:03:16.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:03:16.168 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509453 2020-09-12T14:03:16.168 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:03:16.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:03:16.169 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509453 2020-09-12T14:03:16.169 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:03:16.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509453 2020-09-12T14:03:16.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509453' 2020-09-12T14:03:16.170 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509453 2020-09-12T14:03:16.170 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:03:16.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509453 -lt 64424509453 2020-09-12T14:03:16.593 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:03:16.594 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:03:16.594 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:03:17.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 1 == 0 2020-09-12T14:03:17.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:03:17.160 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:03:17.161 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:03:17.161 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:03:17.161 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:03:17.161 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:03:17.162 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:03:17.558 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:03:17.558 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:03:17.559 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:03:17.559 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:03:18.094 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 1 2020-09-12T14:03:18.094 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T14:03:18.094 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T14:03:18.094 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:678: TEST_divergent_3: get_pg test existing_1 2020-09-12T14:03:18.095 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1031: get_pg: local poolname=test 2020-09-12T14:03:18.095 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1032: get_pg: local objectname=existing_1 2020-09-12T14:03:18.095 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1034: get_pg: ceph --format json osd map test existing_1 2020-09-12T14:03:18.095 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1034: get_pg: jq -r .pgid 2020-09-12T14:03:18.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:678: TEST_divergent_3: local pgid=1.0 2020-09-12T14:03:18.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:681: TEST_divergent_3: echo 'blackholing osds 0 2' 2020-09-12T14:03:18.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:682: TEST_divergent_3: ceph pg dump pgs 2020-09-12T14:03:18.510 INFO:tasks.workunit.client.0.smithi027.stdout:blackholing osds 0 2 2020-09-12T14:03:18.899 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:03:18.899 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 0 0 0 0 48300 0 0 16 16 active+clean 2020-09-12T14:02:46.938591+0000 31'100 31:111 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:03:18.900 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:03:18.900 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:03:18.901 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:03:18.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:683: TEST_divergent_3: for i in $non_divergent 2020-09-12T14:03:18.911 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:685: TEST_divergent_3: get_asok_path osd.0 2020-09-12T14:03:18.911 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2020-09-12T14:03:18.911 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2020-09-12T14:03:18.911 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:03:18.911 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:03:18.912 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:03:18.912 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.0.asok 2020-09-12T14:03:18.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:685: TEST_divergent_3: CEPH_ARGS= 2020-09-12T14:03:18.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:685: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.0.asok config set objectstore_blackhole 1 2020-09-12T14:03:19.013 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:03:19.013 INFO:tasks.workunit.client.0.smithi027.stdout: "success": "objectstore_blackhole = 'true' (not observed, change may require restart) " 2020-09-12T14:03:19.013 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:03:19.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:683: TEST_divergent_3: for i in $non_divergent 2020-09-12T14:03:19.022 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:685: TEST_divergent_3: get_asok_path osd.2 2020-09-12T14:03:19.022 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2020-09-12T14:03:19.023 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2020-09-12T14:03:19.023 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:03:19.023 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:03:19.023 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:03:19.023 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.2.asok 2020-09-12T14:03:19.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:685: TEST_divergent_3: CEPH_ARGS= 2020-09-12T14:03:19.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:685: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.2.asok config set objectstore_blackhole 1 2020-09-12T14:03:19.124 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:03:19.125 INFO:tasks.workunit.client.0.smithi027.stdout: "success": "objectstore_blackhole = 'true' (not observed, change may require restart) " 2020-09-12T14:03:19.125 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:03:19.132 INFO:tasks.workunit.client.0.smithi027.stdout:create new divergent objects 2020-09-12T14:03:19.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:689: TEST_divergent_3: echo 'create new divergent objects' 2020-09-12T14:03:19.133 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: seq 1 2 2020-09-12T14:03:19.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2020-09-12T14:03:19.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2020-09-12T14:03:19.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:692: TEST_divergent_3: rados -p test create newobject_1 2020-09-12T14:03:19.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:695: TEST_divergent_3: echo 'writing divergent objects' 2020-09-12T14:03:19.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:692: TEST_divergent_3: rados -p test create newobject_2 2020-09-12T14:03:19.134 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:696: TEST_divergent_3: seq 1 2 2020-09-12T14:03:19.135 INFO:tasks.workunit.client.0.smithi027.stdout:writing divergent objects 2020-09-12T14:03:19.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:696: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_WRITE) 2020-09-12T14:03:19.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:696: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_WRITE) 2020-09-12T14:03:19.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:698: TEST_divergent_3: rados -p test put existing_1 /etc/resolv.conf 2020-09-12T14:03:19.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:701: TEST_divergent_3: echo 'remove divergent objects' 2020-09-12T14:03:19.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:698: TEST_divergent_3: rados -p test put existing_2 /etc/resolv.conf 2020-09-12T14:03:19.136 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:702: TEST_divergent_3: seq 1 2 2020-09-12T14:03:19.136 INFO:tasks.workunit.client.0.smithi027.stdout:remove divergent objects 2020-09-12T14:03:19.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:702: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_REMOVE) 2020-09-12T14:03:19.137 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:704: TEST_divergent_3: expr 1 + 2 2020-09-12T14:03:19.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:704: TEST_divergent_3: rmi=3 2020-09-12T14:03:19.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:702: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_REMOVE) 2020-09-12T14:03:19.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:705: TEST_divergent_3: rados -p test rm existing_3 2020-09-12T14:03:19.138 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:704: TEST_divergent_3: expr 2 + 2 2020-09-12T14:03:19.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:704: TEST_divergent_3: rmi=4 2020-09-12T14:03:19.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:707: TEST_divergent_3: sleep 10 2020-09-12T14:03:19.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:705: TEST_divergent_3: rados -p test rm existing_4 2020-09-12T14:03:29.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:708: TEST_divergent_3: killall -9 rados 2020-09-12T14:03:29.146 INFO:tasks.workunit.client.0.smithi027.stderr:rados: no process found 2020-09-12T14:03:29.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:711: TEST_divergent_3: echo 'killing all the osds' 2020-09-12T14:03:29.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:712: TEST_divergent_3: ceph pg dump pgs 2020-09-12T14:03:29.147 INFO:tasks.workunit.client.0.smithi027.stdout:killing all the osds 2020-09-12T14:03:29.548 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:03:29.548 INFO:tasks.workunit.client.0.smithi027.stdout:1.1f 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.548 INFO:tasks.workunit.client.0.smithi027.stdout:1.1e 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.548 INFO:tasks.workunit.client.0.smithi027.stdout:1.1d 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.549 INFO:tasks.workunit.client.0.smithi027.stdout:1.1c 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.549 INFO:tasks.workunit.client.0.smithi027.stdout:1.1b 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.549 INFO:tasks.workunit.client.0.smithi027.stdout:1.1a 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.549 INFO:tasks.workunit.client.0.smithi027.stdout:1.19 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.549 INFO:tasks.workunit.client.0.smithi027.stdout:1.18 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.549 INFO:tasks.workunit.client.0.smithi027.stdout:1.17 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.550 INFO:tasks.workunit.client.0.smithi027.stdout:1.16 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.550 INFO:tasks.workunit.client.0.smithi027.stdout:1.15 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.550 INFO:tasks.workunit.client.0.smithi027.stdout:1.14 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.550 INFO:tasks.workunit.client.0.smithi027.stdout:1.13 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.550 INFO:tasks.workunit.client.0.smithi027.stdout:1.12 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.550 INFO:tasks.workunit.client.0.smithi027.stdout:1.11 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.550 INFO:tasks.workunit.client.0.smithi027.stdout:1.10 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.551 INFO:tasks.workunit.client.0.smithi027.stdout:1.5 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.551 INFO:tasks.workunit.client.0.smithi027.stdout:1.4 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.551 INFO:tasks.workunit.client.0.smithi027.stdout:1.3 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.551 INFO:tasks.workunit.client.0.smithi027.stdout:1.2 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.551 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 3 0 0 0 0 1457 0 0 16 16 peering 2020-09-12T14:03:24.928740+0000 31'106 33:121 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:03:29.551 INFO:tasks.workunit.client.0.smithi027.stdout:1.1 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.551 INFO:tasks.workunit.client.0.smithi027.stdout:1.6 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.552 INFO:tasks.workunit.client.0.smithi027.stdout:1.7 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.552 INFO:tasks.workunit.client.0.smithi027.stdout:1.8 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.552 INFO:tasks.workunit.client.0.smithi027.stdout:1.9 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.552 INFO:tasks.workunit.client.0.smithi027.stdout:1.a 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.552 INFO:tasks.workunit.client.0.smithi027.stdout:1.b 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.552 INFO:tasks.workunit.client.0.smithi027.stdout:1.c 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.552 INFO:tasks.workunit.client.0.smithi027.stdout:1.d 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.552 INFO:tasks.workunit.client.0.smithi027.stdout:1.e 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.553 INFO:tasks.workunit.client.0.smithi027.stdout:1.f 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:29.553 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:03:29.553 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:03:29.553 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:03:29.558 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:713: TEST_divergent_3: kill_daemons td/divergent-priors KILL osd 2020-09-12T14:03:29.559 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:03:29.559 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:03:29.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:03:29.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:03:29.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:03:29.665 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:03:29.666 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:714: TEST_divergent_3: for i in $osds 2020-09-12T14:03:29.666 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:716: TEST_divergent_3: ceph osd down osd.0 2020-09-12T14:03:30.278 INFO:tasks.workunit.client.0.smithi027.stderr:osd.0 is already down. 2020-09-12T14:03:30.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:714: TEST_divergent_3: for i in $osds 2020-09-12T14:03:30.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:716: TEST_divergent_3: ceph osd down osd.1 2020-09-12T14:03:30.843 INFO:tasks.workunit.client.0.smithi027.stderr:osd.1 is already down. 2020-09-12T14:03:30.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:714: TEST_divergent_3: for i in $osds 2020-09-12T14:03:30.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:716: TEST_divergent_3: ceph osd down osd.2 2020-09-12T14:03:31.499 INFO:tasks.workunit.client.0.smithi027.stderr:osd.2 is already down. 2020-09-12T14:03:31.508 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:718: TEST_divergent_3: for i in $non_divergent 2020-09-12T14:03:31.508 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:720: TEST_divergent_3: ceph osd out osd.0 2020-09-12T14:03:32.108 INFO:tasks.workunit.client.0.smithi027.stderr:osd.0 is already out. 2020-09-12T14:03:32.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:718: TEST_divergent_3: for i in $non_divergent 2020-09-12T14:03:32.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:720: TEST_divergent_3: ceph osd out osd.2 2020-09-12T14:03:32.686 INFO:tasks.workunit.client.0.smithi027.stderr:osd.2 is already out. 2020-09-12T14:03:32.696 INFO:tasks.workunit.client.0.smithi027.stdout:bringing up non_divergent 0 2 2020-09-12T14:03:32.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:724: TEST_divergent_3: echo 'bringing up non_divergent 0 2' 2020-09-12T14:03:32.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:725: TEST_divergent_3: ceph pg dump pgs 2020-09-12T14:03:33.112 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:03:33.112 INFO:tasks.workunit.client.0.smithi027.stdout:1.1f 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.112 INFO:tasks.workunit.client.0.smithi027.stdout:1.1e 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.113 INFO:tasks.workunit.client.0.smithi027.stdout:1.1d 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.113 INFO:tasks.workunit.client.0.smithi027.stdout:1.1c 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.113 INFO:tasks.workunit.client.0.smithi027.stdout:1.1b 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.113 INFO:tasks.workunit.client.0.smithi027.stdout:1.1a 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.113 INFO:tasks.workunit.client.0.smithi027.stdout:1.19 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.113 INFO:tasks.workunit.client.0.smithi027.stdout:1.18 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.113 INFO:tasks.workunit.client.0.smithi027.stdout:1.17 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.114 INFO:tasks.workunit.client.0.smithi027.stdout:1.16 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.114 INFO:tasks.workunit.client.0.smithi027.stdout:1.15 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.114 INFO:tasks.workunit.client.0.smithi027.stdout:1.14 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.114 INFO:tasks.workunit.client.0.smithi027.stdout:1.13 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.114 INFO:tasks.workunit.client.0.smithi027.stdout:1.12 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.114 INFO:tasks.workunit.client.0.smithi027.stdout:1.11 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.114 INFO:tasks.workunit.client.0.smithi027.stdout:1.10 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.115 INFO:tasks.workunit.client.0.smithi027.stdout:1.5 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.115 INFO:tasks.workunit.client.0.smithi027.stdout:1.4 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.115 INFO:tasks.workunit.client.0.smithi027.stdout:1.3 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.115 INFO:tasks.workunit.client.0.smithi027.stdout:1.2 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.115 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 3 0 0 0 0 1457 0 0 16 16 stale+peering 2020-09-12T14:03:24.928740+0000 31'106 33:121 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:03:33.115 INFO:tasks.workunit.client.0.smithi027.stdout:1.1 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.115 INFO:tasks.workunit.client.0.smithi027.stdout:1.6 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.116 INFO:tasks.workunit.client.0.smithi027.stdout:1.7 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.116 INFO:tasks.workunit.client.0.smithi027.stdout:1.8 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.116 INFO:tasks.workunit.client.0.smithi027.stdout:1.9 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.116 INFO:tasks.workunit.client.0.smithi027.stdout:1.a 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.116 INFO:tasks.workunit.client.0.smithi027.stdout:1.b 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.116 INFO:tasks.workunit.client.0.smithi027.stdout:1.c 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.116 INFO:tasks.workunit.client.0.smithi027.stdout:1.d 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.116 INFO:tasks.workunit.client.0.smithi027.stdout:1.e 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.117 INFO:tasks.workunit.client.0.smithi027.stdout:1.f 0 0 0 0 0 0 0 0 0 0 unknown 2020-09-12T14:03:24.731600+0000 0'0 0:0 [] -1 [] -1 0'0 2020-09-12T14:03:24.731600+0000 0'0 2020-09-12T14:03:24.731600+0000 0 2020-09-12T14:03:33.117 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:03:33.117 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:03:33.117 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:03:33.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:726: TEST_divergent_3: for i in $non_divergent 2020-09-12T14:03:33.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:728: TEST_divergent_3: activate_osd td/divergent-priors 0 2020-09-12T14:03:33.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/divergent-priors 2020-09-12T14:03:33.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T14:03:33.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=0 2020-09-12T14:03:33.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T14:03:33.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/divergent-priors/0 2020-09-12T14:03:33.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T14:03:33.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:03:33.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:03:33.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:03:33.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2020-09-12T14:03:33.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2020-09-12T14:03:33.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T14:03:33.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T14:03:33.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T14:03:33.127 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T14:03:33.127 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:03:33.127 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:03:33.127 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:03:33.127 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:03:33.127 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:03:33.128 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:03:33.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:03:33.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:03:33.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T14:03:33.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T14:03:33.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:03:33.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:03:33.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:03:33.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T14:03:33.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T14:03:33.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/divergent-priors/0 2020-09-12T14:03:33.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.0 2020-09-12T14:03:33.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 0 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:03:33.132 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/divergent-priors/0/whoami 2020-09-12T14:03:33.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 0 = 0 ']' 2020-09-12T14:03:33.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T14:03:33.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T14:03:33.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T14:03:33.132 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.0 2020-09-12T14:03:33.155 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:03:33.154+0000 7f3322e99ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:03:33.157 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:03:33.156+0000 7f3322e99ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:03:33.158 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:03:33.157+0000 7f3322e99ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:03:33.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: wait_for_osd up 0 2020-09-12T14:03:33.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:03:33.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T14:03:33.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:03:33.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:03:33.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:03:33.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:03:33.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:03:33.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:03:33.542 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:03:33.678 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:03:33.677+0000 7f3322e99ec0 -1 Falling back to public interface 2020-09-12T14:03:33.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:03:34.906 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:03:34.905+0000 7f3322e99ec0 -1 osd.0 31 log_to_monitors {default=true} 2020-09-12T14:03:34.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:03:34.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:03:34.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:03:34.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:03:34.958 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:03:34.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:03:35.456 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up out weight 0 up_from 40 up_thru 0 down_at 35 last_clean_interval [5,31) [v2:127.0.0.1:6802/49081,v1:127.0.0.1:6803/49081] [v2:127.0.0.1:6804/49081,v1:127.0.0.1:6805/49081] exists,up 9d8d2ee9-0710-4a4e-a0ba-cec45cbf29a2 2020-09-12T14:03:35.457 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:03:35.457 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:03:35.457 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:03:35.457 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:726: TEST_divergent_3: for i in $non_divergent 2020-09-12T14:03:35.457 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:728: TEST_divergent_3: activate_osd td/divergent-priors 2 2020-09-12T14:03:35.457 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/divergent-priors 2020-09-12T14:03:35.457 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T14:03:35.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=2 2020-09-12T14:03:35.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T14:03:35.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/divergent-priors/2 2020-09-12T14:03:35.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T14:03:35.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:03:35.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:03:35.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:03:35.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2020-09-12T14:03:35.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2020-09-12T14:03:35.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T14:03:35.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T14:03:35.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T14:03:35.459 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T14:03:35.459 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:03:35.459 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:03:35.460 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:03:35.460 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:03:35.460 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:03:35.460 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:03:35.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:03:35.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:03:35.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T14:03:35.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T14:03:35.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:03:35.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:03:35.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:03:35.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T14:03:35.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T14:03:35.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/divergent-priors/2 2020-09-12T14:03:35.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.2 2020-09-12T14:03:35.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 2 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:03:35.463 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/divergent-priors/2/whoami 2020-09-12T14:03:35.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 2 = 2 ']' 2020-09-12T14:03:35.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T14:03:35.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T14:03:35.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T14:03:35.464 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.2 2020-09-12T14:03:35.487 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:03:35.486+0000 7fdf61ebcec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:03:35.488 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:03:35.488+0000 7fdf61ebcec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:03:35.490 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:03:35.489+0000 7fdf61ebcec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:03:35.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: wait_for_osd up 2 2020-09-12T14:03:35.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:03:35.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T14:03:35.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:03:35.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:03:35.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:03:35.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:03:35.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:03:35.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:03:35.882 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:03:36.009 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:03:36.007+0000 7fdf61ebcec0 -1 Falling back to public interface 2020-09-12T14:03:36.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:03:37.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:03:37.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:03:37.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:03:37.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:03:37.292 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:03:37.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:03:37.429 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:03:37.428+0000 7fdf61ebcec0 -1 osd.2 31 log_to_monitors {default=true} 2020-09-12T14:03:37.740 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up out weight 0 up_from 41 up_thru 0 down_at 37 last_clean_interval [15,31) [v2:127.0.0.1:6810/49480,v1:127.0.0.1:6811/49480] [v2:127.0.0.1:6812/49480,v1:127.0.0.1:6813/49480] exists,up ed2b43d9-2a50-45c4-b6a7-8f00f86304e8 2020-09-12T14:03:37.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:03:37.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:03:37.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:03:37.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:730: TEST_divergent_3: for i in $non_divergent 2020-09-12T14:03:37.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:732: TEST_divergent_3: ceph osd in osd.0 2020-09-12T14:03:38.397 INFO:tasks.workunit.client.0.smithi027.stderr:osd.0 is already in. 2020-09-12T14:03:38.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:730: TEST_divergent_3: for i in $non_divergent 2020-09-12T14:03:38.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:732: TEST_divergent_3: ceph osd in osd.2 2020-09-12T14:03:39.400 INFO:tasks.workunit.client.0.smithi027.stderr:osd.2 is already in. 2020-09-12T14:03:39.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:735: TEST_divergent_3: WAIT_FOR_CLEAN_TIMEOUT=20 2020-09-12T14:03:39.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:735: TEST_divergent_3: wait_for_clean 2020-09-12T14:03:39.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:03:39.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:03:39.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:03:39.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:03:39.411 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 20 .1 2020-09-12T14:03:39.412 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:03:39.412 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:03:39.412 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:03:39.412 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:03:39.413 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:03:39.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:03:39.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:03:39.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:03:39.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:03:39.472 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:03:39.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:03:39.935 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:03:39.935 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T14:03:39.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:03:39.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:03:39.936 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:03:40.086 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=171798691844 2020-09-12T14:03:40.086 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 171798691844 2020-09-12T14:03:40.086 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-171798691844' 2020-09-12T14:03:40.086 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:03:40.087 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:03:40.193 INFO:tasks.workunit.client.0.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-09-12T14:03:40.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq= 2020-09-12T14:03:40.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z '' 2020-09-12T14:03:40.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2075: flush_pg_stats: continue 2020-09-12T14:03:40.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:03:40.195 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:03:40.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=176093659139 2020-09-12T14:03:40.342 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 176093659139 2020-09-12T14:03:40.342 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-171798691844 2-176093659139' 2020-09-12T14:03:40.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:03:40.343 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-171798691844 2020-09-12T14:03:40.344 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:03:40.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:03:40.344 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-171798691844 2020-09-12T14:03:40.345 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:03:40.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=171798691844 2020-09-12T14:03:40.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 171798691844' 2020-09-12T14:03:40.346 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:03:40.346 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 171798691844 2020-09-12T14:03:40.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 1 -lt 171798691844 2020-09-12T14:03:40.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:03:41.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:03:41.787 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:03:42.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 171798691844 -lt 171798691844 2020-09-12T14:03:42.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:03:42.206 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-176093659139 2020-09-12T14:03:42.206 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:03:42.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:03:42.207 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-176093659139 2020-09-12T14:03:42.208 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:03:42.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=176093659139 2020-09-12T14:03:42.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 176093659139' 2020-09-12T14:03:42.209 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:03:42.209 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 176093659139 2020-09-12T14:03:42.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 176093659139 -lt 176093659139 2020-09-12T14:03:42.620 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:03:42.620 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:03:42.620 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:03:43.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 32 == 0 2020-09-12T14:03:43.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:03:43.180 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:03:43.181 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:03:43.181 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:03:43.181 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:03:43.181 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:03:43.181 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:03:43.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:03:43.582 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:03:43.582 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:03:43.583 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:03:44.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 32 2020-09-12T14:03:44.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' -1 2020-09-12T14:03:44.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1598: wait_for_clean: loop=0 2020-09-12T14:03:44.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1599: wait_for_clean: num_active_clean=0 2020-09-12T14:03:44.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:03:44.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:03:44.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:03:44.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:03:44.233 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:03:44.233 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:03:44.234 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:03:44.234 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:03:44.234 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:03:44.234 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:03:44.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:03:44.629 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:03:44.630 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:03:44.630 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:03:45.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 32 2020-09-12T14:03:45.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:03:45.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:03:45.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:03:45.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:03:45.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:03:45.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:03:45.169 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:03:45.169 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:03:45.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:03:45.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:03:45.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 1 >= 8 )) 2020-09-12T14:03:45.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:03:45.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.2 2020-09-12T14:03:45.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:03:45.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:03:45.900 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:03:45.900 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:03:45.900 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:03:45.900 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:03:45.900 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:03:45.901 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:03:46.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:03:46.297 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:03:46.297 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:03:46.297 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:03:46.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 32 2020-09-12T14:03:46.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:03:46.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:03:46.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:03:46.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:03:46.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:03:46.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:03:46.831 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:03:46.831 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:03:47.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:03:47.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:03:47.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 2 >= 8 )) 2020-09-12T14:03:47.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:03:47.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.4 2020-09-12T14:03:47.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:03:47.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:03:47.774 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:03:47.774 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:03:47.774 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:03:47.774 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:03:47.775 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:03:47.775 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:03:48.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:03:48.180 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:03:48.181 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:03:48.181 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:03:48.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 32 2020-09-12T14:03:48.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:03:48.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:03:48.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:03:48.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:03:48.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:03:48.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:03:48.726 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:03:48.727 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:03:49.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:03:49.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:03:49.261 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 3 >= 8 )) 2020-09-12T14:03:49.261 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:03:49.261 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.8 2020-09-12T14:03:50.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:03:50.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:03:50.062 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:03:50.062 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:03:50.062 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:03:50.062 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:03:50.063 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:03:50.063 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:03:50.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:03:50.473 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:03:50.473 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:03:50.474 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:03:51.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 32 2020-09-12T14:03:51.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:03:51.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:03:51.009 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:03:51.009 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:03:51.009 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:03:51.009 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:03:51.009 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:03:51.010 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:03:51.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:03:51.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:03:51.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 4 >= 8 )) 2020-09-12T14:03:51.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:03:51.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 1.6 2020-09-12T14:03:53.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:03:53.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:03:53.145 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:03:53.145 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:03:53.145 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:03:53.145 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:03:53.146 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:03:53.146 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:03:53.557 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:03:53.557 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:03:53.558 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:03:53.558 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:03:54.094 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 32 2020-09-12T14:03:54.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:03:54.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:03:54.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:03:54.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:03:54.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:03:54.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:03:54.096 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:03:54.096 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:03:54.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:03:54.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:03:54.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 5 >= 8 )) 2020-09-12T14:03:54.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:03:54.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 3.2 2020-09-12T14:03:57.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:03:57.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:03:57.838 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:03:57.838 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:03:57.838 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:03:57.839 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:03:57.839 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:03:57.839 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:03:58.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:03:58.253 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:03:58.253 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:03:58.253 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:03:58.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 32 2020-09-12T14:03:58.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:03:58.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:03:58.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:03:58.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:03:58.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:03:58.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:03:58.811 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:03:58.811 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:03:59.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:03:59.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:03:59.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 6 >= 8 )) 2020-09-12T14:03:59.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:03:59.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 6.4 2020-09-12T14:04:05.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:04:05.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:04:05.769 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:04:05.770 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:04:05.770 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:04:05.770 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:04:05.770 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:04:05.770 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:04:06.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:04:06.180 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:04:06.180 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:04:06.181 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:04:06.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 32 2020-09-12T14:04:06.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:04:06.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:04:06.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:04:06.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:04:06.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:04:06.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:04:06.721 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:04:06.721 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:04:07.262 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:04:07.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:04:07.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 7 >= 8 )) 2020-09-12T14:04:07.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:04:07.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 7.3 2020-09-12T14:04:14.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:04:14.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:04:14.564 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:04:14.565 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:04:14.565 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:04:14.565 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:04:14.565 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:04:14.565 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:04:14.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:04:14.978 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:04:14.979 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:04:14.979 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:04:15.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 32 2020-09-12T14:04:15.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:04:15.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:04:15.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:04:15.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:04:15.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:04:15.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:04:15.520 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:04:15.520 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:04:16.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:04:16.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:04:16.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 8 >= 8 )) 2020-09-12T14:04:16.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: wait_for_clean: ceph report 2020-09-12T14:04:16.581 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:04:16.581 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_fingerprint": "62ae0e86-0762-4c2d-b8c6-c318affebcfc", 2020-09-12T14:04:16.581 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:04:16.582 INFO:tasks.workunit.client.0.smithi027.stdout: "commit": "f346cc73d15a568d3bec1c99a58953de76f57b85", 2020-09-12T14:04:16.582 INFO:tasks.workunit.client.0.smithi027.stdout: "timestamp": "2020-09-12T14:04:16.579088+0000", 2020-09-12T14:04:16.582 INFO:tasks.workunit.client.0.smithi027.stdout: "tag": "", 2020-09-12T14:04:16.583 INFO:tasks.workunit.client.0.smithi027.stdout: "health": { 2020-09-12T14:04:16.583 INFO:tasks.workunit.client.0.smithi027.stdout: "status": "HEALTH_WARN", 2020-09-12T14:04:16.583 INFO:tasks.workunit.client.0.smithi027.stdout: "checks": { 2020-09-12T14:04:16.583 INFO:tasks.workunit.client.0.smithi027.stdout: "OSDMAP_FLAGS": { 2020-09-12T14:04:16.584 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:04:16.584 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:04:16.584 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "nodown,noout,noin flag(s) set", 2020-09-12T14:04:16.584 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 17 2020-09-12T14:04:16.585 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.585 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [], 2020-09-12T14:04:16.585 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:04:16.585 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.586 INFO:tasks.workunit.client.0.smithi027.stdout: "OSD_DOWN": { 2020-09-12T14:04:16.586 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:04:16.586 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:04:16.586 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "1 osds down", 2020-09-12T14:04:16.587 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 1 2020-09-12T14:04:16.587 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.587 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [ 2020-09-12T14:04:16.587 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.588 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "osd.1 (root=default,host=smithi027) is down" 2020-09-12T14:04:16.588 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.588 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:16.588 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:04:16.589 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.589 INFO:tasks.workunit.client.0.smithi027.stdout: "PG_DEGRADED": { 2020-09-12T14:04:16.589 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:04:16.590 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:04:16.590 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 32 pgs degraded", 2020-09-12T14:04:16.590 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 32 2020-09-12T14:04:16.590 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.591 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [ 2020-09-12T14:04:16.591 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.591 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.591 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.592 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.592 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.1 is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.592 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.592 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.593 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.2 is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.593 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.593 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.594 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.3 is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.594 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.594 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.595 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.4 is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.595 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.595 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.596 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.5 is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.596 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.596 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.597 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.6 is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.597 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.597 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.597 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.7 is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.598 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.598 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.598 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.8 is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.598 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.599 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.599 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.9 is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.599 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.599 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.600 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.a is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.600 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.600 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.600 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.b is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.601 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.601 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.601 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.c is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.602 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.602 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.602 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.d is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.602 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.603 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.603 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.e is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.603 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.603 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.604 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.f is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.604 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.604 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.604 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.10 is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.605 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.605 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.605 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.11 is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.606 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.606 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.606 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.12 is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.607 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.607 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.607 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.13 is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.608 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.608 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.608 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.14 is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.608 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.609 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.609 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.15 is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.609 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.609 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.609 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.16 is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.610 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.610 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.610 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.17 is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.610 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.610 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.610 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.18 is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.611 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.611 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.611 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.19 is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.611 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.611 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.611 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.1a is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.612 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.612 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.612 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.1b is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.612 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.612 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.613 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.1c is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.613 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.613 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.613 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.1d is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.613 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.613 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.614 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.1e is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.614 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.614 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.614 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.1f is active+undersized+degraded, acting [0,2]" 2020-09-12T14:04:16.614 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.614 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:16.615 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:04:16.615 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.615 INFO:tasks.workunit.client.0.smithi027.stdout: "POOL_APP_NOT_ENABLED": { 2020-09-12T14:04:16.615 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:04:16.615 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:04:16.615 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "1 pool(s) do not have an application enabled", 2020-09-12T14:04:16.615 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 1 2020-09-12T14:04:16.615 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.616 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [ 2020-09-12T14:04:16.616 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.616 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "application not enabled on pool 'test'" 2020-09-12T14:04:16.616 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.616 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.616 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2020-09-12T14:04:16.616 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.617 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:16.617 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:04:16.617 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.617 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.617 INFO:tasks.workunit.client.0.smithi027.stdout: "mutes": [] 2020-09-12T14:04:16.617 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.618 INFO:tasks.workunit.client.0.smithi027.stdout: "monmap_first_committed": 1, 2020-09-12T14:04:16.618 INFO:tasks.workunit.client.0.smithi027.stdout: "monmap_last_committed": 1, 2020-09-12T14:04:16.618 INFO:tasks.workunit.client.0.smithi027.stdout: "monmap": { 2020-09-12T14:04:16.618 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 1, 2020-09-12T14:04:16.618 INFO:tasks.workunit.client.0.smithi027.stdout: "fsid": "7e90bad3-7973-4a8c-be65-6326ab931fe8", 2020-09-12T14:04:16.618 INFO:tasks.workunit.client.0.smithi027.stdout: "modified": "2020-09-12T14:02:17.274059Z", 2020-09-12T14:04:16.618 INFO:tasks.workunit.client.0.smithi027.stdout: "created": "2020-09-12T14:02:17.274059Z", 2020-09-12T14:04:16.618 INFO:tasks.workunit.client.0.smithi027.stdout: "min_mon_release": 16, 2020-09-12T14:04:16.619 INFO:tasks.workunit.client.0.smithi027.stdout: "min_mon_release_name": "pacific", 2020-09-12T14:04:16.619 INFO:tasks.workunit.client.0.smithi027.stdout: "features": { 2020-09-12T14:04:16.619 INFO:tasks.workunit.client.0.smithi027.stdout: "persistent": [ 2020-09-12T14:04:16.619 INFO:tasks.workunit.client.0.smithi027.stdout: "kraken", 2020-09-12T14:04:16.619 INFO:tasks.workunit.client.0.smithi027.stdout: "luminous", 2020-09-12T14:04:16.619 INFO:tasks.workunit.client.0.smithi027.stdout: "mimic", 2020-09-12T14:04:16.619 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap-prune", 2020-09-12T14:04:16.620 INFO:tasks.workunit.client.0.smithi027.stdout: "nautilus", 2020-09-12T14:04:16.620 INFO:tasks.workunit.client.0.smithi027.stdout: "octopus", 2020-09-12T14:04:16.620 INFO:tasks.workunit.client.0.smithi027.stdout: "pacific" 2020-09-12T14:04:16.620 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:16.620 INFO:tasks.workunit.client.0.smithi027.stdout: "optional": [] 2020-09-12T14:04:16.620 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.620 INFO:tasks.workunit.client.0.smithi027.stdout: "mons": [ 2020-09-12T14:04:16.620 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.621 INFO:tasks.workunit.client.0.smithi027.stdout: "rank": 0, 2020-09-12T14:04:16.621 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "a", 2020-09-12T14:04:16.621 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:04:16.621 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:16.621 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.621 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:16.621 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:7115", 2020-09-12T14:04:16.622 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 0 2020-09-12T14:04:16.622 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.622 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.622 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.622 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:7115/0", 2020-09-12T14:04:16.622 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:7115/0", 2020-09-12T14:04:16.622 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 0, 2020-09-12T14:04:16.623 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 0 2020-09-12T14:04:16.623 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.623 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.623 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.623 INFO:tasks.workunit.client.0.smithi027.stdout: "quorum": [ 2020-09-12T14:04:16.623 INFO:tasks.workunit.client.0.smithi027.stdout: 0 2020-09-12T14:04:16.623 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:16.623 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap": { 2020-09-12T14:04:16.624 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 47, 2020-09-12T14:04:16.624 INFO:tasks.workunit.client.0.smithi027.stdout: "fsid": "7e90bad3-7973-4a8c-be65-6326ab931fe8", 2020-09-12T14:04:16.624 INFO:tasks.workunit.client.0.smithi027.stdout: "created": "2020-09-12T14:02:18.091340+0000", 2020-09-12T14:04:16.624 INFO:tasks.workunit.client.0.smithi027.stdout: "modified": "2020-09-12T14:03:39.453198+0000", 2020-09-12T14:04:16.624 INFO:tasks.workunit.client.0.smithi027.stdout: "last_up_change": "2020-09-12T14:03:37.501958+0000", 2020-09-12T14:04:16.624 INFO:tasks.workunit.client.0.smithi027.stdout: "last_in_change": "2020-09-12T14:03:38.883551+0000", 2020-09-12T14:04:16.624 INFO:tasks.workunit.client.0.smithi027.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-09-12T14:04:16.625 INFO:tasks.workunit.client.0.smithi027.stdout: "flags_num": 5800384, 2020-09-12T14:04:16.625 INFO:tasks.workunit.client.0.smithi027.stdout: "flags_set": [ 2020-09-12T14:04:16.625 INFO:tasks.workunit.client.0.smithi027.stdout: "nodown", 2020-09-12T14:04:16.625 INFO:tasks.workunit.client.0.smithi027.stdout: "noin", 2020-09-12T14:04:16.625 INFO:tasks.workunit.client.0.smithi027.stdout: "noout", 2020-09-12T14:04:16.625 INFO:tasks.workunit.client.0.smithi027.stdout: "pglog_hardlimit", 2020-09-12T14:04:16.625 INFO:tasks.workunit.client.0.smithi027.stdout: "purged_snapdirs", 2020-09-12T14:04:16.625 INFO:tasks.workunit.client.0.smithi027.stdout: "recovery_deletes", 2020-09-12T14:04:16.626 INFO:tasks.workunit.client.0.smithi027.stdout: "sortbitwise" 2020-09-12T14:04:16.626 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:16.626 INFO:tasks.workunit.client.0.smithi027.stdout: "crush_version": 7, 2020-09-12T14:04:16.626 INFO:tasks.workunit.client.0.smithi027.stdout: "full_ratio": 0.99000000953674316, 2020-09-12T14:04:16.626 INFO:tasks.workunit.client.0.smithi027.stdout: "backfillfull_ratio": 0.99000000953674316, 2020-09-12T14:04:16.626 INFO:tasks.workunit.client.0.smithi027.stdout: "nearfull_ratio": 0.85000002384185791, 2020-09-12T14:04:16.627 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_snapshot": "", 2020-09-12T14:04:16.627 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_max": 1, 2020-09-12T14:04:16.627 INFO:tasks.workunit.client.0.smithi027.stdout: "max_osd": 3, 2020-09-12T14:04:16.627 INFO:tasks.workunit.client.0.smithi027.stdout: "require_min_compat_client": "luminous", 2020-09-12T14:04:16.627 INFO:tasks.workunit.client.0.smithi027.stdout: "min_compat_client": "jewel", 2020-09-12T14:04:16.627 INFO:tasks.workunit.client.0.smithi027.stdout: "require_osd_release": "pacific", 2020-09-12T14:04:16.627 INFO:tasks.workunit.client.0.smithi027.stdout: "pools": [ 2020-09-12T14:04:16.627 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.628 INFO:tasks.workunit.client.0.smithi027.stdout: "pool": 1, 2020-09-12T14:04:16.628 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_name": "test", 2020-09-12T14:04:16.628 INFO:tasks.workunit.client.0.smithi027.stdout: "create_time": "2020-09-12T14:02:45.510238+0000", 2020-09-12T14:04:16.628 INFO:tasks.workunit.client.0.smithi027.stdout: "flags": 1, 2020-09-12T14:04:16.628 INFO:tasks.workunit.client.0.smithi027.stdout: "flags_names": "hashpspool", 2020-09-12T14:04:16.628 INFO:tasks.workunit.client.0.smithi027.stdout: "type": 1, 2020-09-12T14:04:16.628 INFO:tasks.workunit.client.0.smithi027.stdout: "size": 3, 2020-09-12T14:04:16.629 INFO:tasks.workunit.client.0.smithi027.stdout: "min_size": 2, 2020-09-12T14:04:16.629 INFO:tasks.workunit.client.0.smithi027.stdout: "crush_rule": 0, 2020-09-12T14:04:16.629 INFO:tasks.workunit.client.0.smithi027.stdout: "object_hash": 2, 2020-09-12T14:04:16.629 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_autoscale_mode": "on", 2020-09-12T14:04:16.629 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_num": 32, 2020-09-12T14:04:16.629 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_placement_num": 1, 2020-09-12T14:04:16.629 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_placement_num_target": 32, 2020-09-12T14:04:16.630 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_num_target": 32, 2020-09-12T14:04:16.630 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_num_pending": 32, 2020-09-12T14:04:16.630 INFO:tasks.workunit.client.0.smithi027.stdout: "last_pg_merge_meta": { 2020-09-12T14:04:16.630 INFO:tasks.workunit.client.0.smithi027.stdout: "source_pgid": "0.0", 2020-09-12T14:04:16.630 INFO:tasks.workunit.client.0.smithi027.stdout: "ready_epoch": 0, 2020-09-12T14:04:16.630 INFO:tasks.workunit.client.0.smithi027.stdout: "last_epoch_started": 0, 2020-09-12T14:04:16.631 INFO:tasks.workunit.client.0.smithi027.stdout: "last_epoch_clean": 0, 2020-09-12T14:04:16.631 INFO:tasks.workunit.client.0.smithi027.stdout: "source_version": "0'0", 2020-09-12T14:04:16.631 INFO:tasks.workunit.client.0.smithi027.stdout: "target_version": "0'0" 2020-09-12T14:04:16.631 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.631 INFO:tasks.workunit.client.0.smithi027.stdout: "last_change": "33", 2020-09-12T14:04:16.631 INFO:tasks.workunit.client.0.smithi027.stdout: "last_force_op_resend": "0", 2020-09-12T14:04:16.631 INFO:tasks.workunit.client.0.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2020-09-12T14:04:16.632 INFO:tasks.workunit.client.0.smithi027.stdout: "last_force_op_resend_preluminous": "33", 2020-09-12T14:04:16.632 INFO:tasks.workunit.client.0.smithi027.stdout: "auid": 0, 2020-09-12T14:04:16.632 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_mode": "selfmanaged", 2020-09-12T14:04:16.632 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_seq": 0, 2020-09-12T14:04:16.632 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_epoch": 0, 2020-09-12T14:04:16.632 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_snaps": [], 2020-09-12T14:04:16.632 INFO:tasks.workunit.client.0.smithi027.stdout: "removed_snaps": "[]", 2020-09-12T14:04:16.633 INFO:tasks.workunit.client.0.smithi027.stdout: "quota_max_bytes": 0, 2020-09-12T14:04:16.633 INFO:tasks.workunit.client.0.smithi027.stdout: "quota_max_objects": 0, 2020-09-12T14:04:16.633 INFO:tasks.workunit.client.0.smithi027.stdout: "tiers": [], 2020-09-12T14:04:16.633 INFO:tasks.workunit.client.0.smithi027.stdout: "tier_of": -1, 2020-09-12T14:04:16.633 INFO:tasks.workunit.client.0.smithi027.stdout: "read_tier": -1, 2020-09-12T14:04:16.633 INFO:tasks.workunit.client.0.smithi027.stdout: "write_tier": -1, 2020-09-12T14:04:16.633 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_mode": "none", 2020-09-12T14:04:16.634 INFO:tasks.workunit.client.0.smithi027.stdout: "target_max_bytes": 0, 2020-09-12T14:04:16.634 INFO:tasks.workunit.client.0.smithi027.stdout: "target_max_objects": 0, 2020-09-12T14:04:16.634 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-09-12T14:04:16.634 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-09-12T14:04:16.634 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2020-09-12T14:04:16.634 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_min_flush_age": 0, 2020-09-12T14:04:16.634 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_min_evict_age": 0, 2020-09-12T14:04:16.635 INFO:tasks.workunit.client.0.smithi027.stdout: "erasure_code_profile": "", 2020-09-12T14:04:16.635 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_params": { 2020-09-12T14:04:16.635 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "none" 2020-09-12T14:04:16.635 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.635 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_period": 0, 2020-09-12T14:04:16.635 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_count": 0, 2020-09-12T14:04:16.636 INFO:tasks.workunit.client.0.smithi027.stdout: "use_gmt_hitset": true, 2020-09-12T14:04:16.636 INFO:tasks.workunit.client.0.smithi027.stdout: "min_read_recency_for_promote": 0, 2020-09-12T14:04:16.636 INFO:tasks.workunit.client.0.smithi027.stdout: "min_write_recency_for_promote": 0, 2020-09-12T14:04:16.636 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2020-09-12T14:04:16.636 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_search_last_n": 0, 2020-09-12T14:04:16.636 INFO:tasks.workunit.client.0.smithi027.stdout: "grade_table": [], 2020-09-12T14:04:16.636 INFO:tasks.workunit.client.0.smithi027.stdout: "stripe_width": 0, 2020-09-12T14:04:16.636 INFO:tasks.workunit.client.0.smithi027.stdout: "expected_num_objects": 0, 2020-09-12T14:04:16.637 INFO:tasks.workunit.client.0.smithi027.stdout: "fast_read": false, 2020-09-12T14:04:16.637 INFO:tasks.workunit.client.0.smithi027.stdout: "options": {}, 2020-09-12T14:04:16.637 INFO:tasks.workunit.client.0.smithi027.stdout: "application_metadata": {} 2020-09-12T14:04:16.637 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.637 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:16.637 INFO:tasks.workunit.client.0.smithi027.stdout: "osds": [ 2020-09-12T14:04:16.638 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.638 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:04:16.638 INFO:tasks.workunit.client.0.smithi027.stdout: "uuid": "9d8d2ee9-0710-4a4e-a0ba-cec45cbf29a2", 2020-09-12T14:04:16.638 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 1, 2020-09-12T14:04:16.638 INFO:tasks.workunit.client.0.smithi027.stdout: "in": 1, 2020-09-12T14:04:16.638 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 1, 2020-09-12T14:04:16.638 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_affinity": 1, 2020-09-12T14:04:16.639 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_begin": 5, 2020-09-12T14:04:16.639 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_end": 31, 2020-09-12T14:04:16.639 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 40, 2020-09-12T14:04:16.639 INFO:tasks.workunit.client.0.smithi027.stdout: "up_thru": 45, 2020-09-12T14:04:16.639 INFO:tasks.workunit.client.0.smithi027.stdout: "down_at": 35, 2020-09-12T14:04:16.639 INFO:tasks.workunit.client.0.smithi027.stdout: "lost_at": 0, 2020-09-12T14:04:16.639 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:04:16.640 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:16.640 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.640 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:16.640 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6802", 2020-09-12T14:04:16.640 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49081 2020-09-12T14:04:16.640 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.640 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.641 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:16.641 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6803", 2020-09-12T14:04:16.641 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49081 2020-09-12T14:04:16.641 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.641 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.642 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.642 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addrs": { 2020-09-12T14:04:16.642 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:16.642 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.642 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:16.642 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6804", 2020-09-12T14:04:16.642 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49081 2020-09-12T14:04:16.643 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.643 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.643 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:16.643 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6805", 2020-09-12T14:04:16.643 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49081 2020-09-12T14:04:16.643 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.643 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.643 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.644 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addrs": { 2020-09-12T14:04:16.644 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:16.644 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.644 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:16.644 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6808", 2020-09-12T14:04:16.644 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49081 2020-09-12T14:04:16.644 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.644 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.645 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:16.645 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6809", 2020-09-12T14:04:16.645 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49081 2020-09-12T14:04:16.645 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.645 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.645 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.645 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addrs": { 2020-09-12T14:04:16.646 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:16.646 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.646 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:16.646 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6806", 2020-09-12T14:04:16.646 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49081 2020-09-12T14:04:16.646 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.646 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.647 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:16.647 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6807", 2020-09-12T14:04:16.647 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49081 2020-09-12T14:04:16.647 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.647 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.647 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.647 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:6803/49081", 2020-09-12T14:04:16.648 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addr": "127.0.0.1:6805/49081", 2020-09-12T14:04:16.648 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addr": "127.0.0.1:6809/49081", 2020-09-12T14:04:16.648 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addr": "127.0.0.1:6807/49081", 2020-09-12T14:04:16.648 INFO:tasks.workunit.client.0.smithi027.stdout: "state": [ 2020-09-12T14:04:16.648 INFO:tasks.workunit.client.0.smithi027.stdout: "exists", 2020-09-12T14:04:16.648 INFO:tasks.workunit.client.0.smithi027.stdout: "up" 2020-09-12T14:04:16.649 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.649 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.649 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.649 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:04:16.649 INFO:tasks.workunit.client.0.smithi027.stdout: "uuid": "06975646-fe2b-4ca5-82e0-a2d0e01e06a8", 2020-09-12T14:04:16.649 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 0, 2020-09-12T14:04:16.649 INFO:tasks.workunit.client.0.smithi027.stdout: "in": 1, 2020-09-12T14:04:16.650 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 1, 2020-09-12T14:04:16.650 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_affinity": 1, 2020-09-12T14:04:16.650 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_begin": 0, 2020-09-12T14:04:16.650 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_end": 0, 2020-09-12T14:04:16.650 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 10, 2020-09-12T14:04:16.650 INFO:tasks.workunit.client.0.smithi027.stdout: "up_thru": 33, 2020-09-12T14:04:16.650 INFO:tasks.workunit.client.0.smithi027.stdout: "down_at": 36, 2020-09-12T14:04:16.651 INFO:tasks.workunit.client.0.smithi027.stdout: "lost_at": 0, 2020-09-12T14:04:16.651 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:04:16.651 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:16.651 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.651 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:16.651 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6810", 2020-09-12T14:04:16.651 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 44126 2020-09-12T14:04:16.651 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.652 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.652 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:16.652 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6811", 2020-09-12T14:04:16.652 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 44126 2020-09-12T14:04:16.652 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.652 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.652 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.652 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addrs": { 2020-09-12T14:04:16.653 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:16.653 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.653 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:16.653 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6812", 2020-09-12T14:04:16.653 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 44126 2020-09-12T14:04:16.653 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.653 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.654 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:16.654 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6813", 2020-09-12T14:04:16.654 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 44126 2020-09-12T14:04:16.654 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.654 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.654 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.654 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addrs": { 2020-09-12T14:04:16.654 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:16.655 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.655 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:16.655 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6816", 2020-09-12T14:04:16.655 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 44126 2020-09-12T14:04:16.655 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.655 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.655 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:16.655 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6817", 2020-09-12T14:04:16.656 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 44126 2020-09-12T14:04:16.656 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.656 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.656 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.656 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addrs": { 2020-09-12T14:04:16.656 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:16.656 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.656 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:16.657 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6814", 2020-09-12T14:04:16.657 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 44126 2020-09-12T14:04:16.657 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.657 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.657 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:16.657 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6815", 2020-09-12T14:04:16.657 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 44126 2020-09-12T14:04:16.657 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.658 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.658 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.658 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:6811/44126", 2020-09-12T14:04:16.658 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addr": "127.0.0.1:6813/44126", 2020-09-12T14:04:16.658 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addr": "127.0.0.1:6817/44126", 2020-09-12T14:04:16.658 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addr": "127.0.0.1:6815/44126", 2020-09-12T14:04:16.658 INFO:tasks.workunit.client.0.smithi027.stdout: "state": [ 2020-09-12T14:04:16.659 INFO:tasks.workunit.client.0.smithi027.stdout: "exists" 2020-09-12T14:04:16.659 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.659 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.659 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.659 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:04:16.660 INFO:tasks.workunit.client.0.smithi027.stdout: "uuid": "ed2b43d9-2a50-45c4-b6a7-8f00f86304e8", 2020-09-12T14:04:16.660 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 1, 2020-09-12T14:04:16.660 INFO:tasks.workunit.client.0.smithi027.stdout: "in": 1, 2020-09-12T14:04:16.660 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 1, 2020-09-12T14:04:16.660 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_affinity": 1, 2020-09-12T14:04:16.660 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_begin": 15, 2020-09-12T14:04:16.660 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_end": 31, 2020-09-12T14:04:16.660 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 41, 2020-09-12T14:04:16.661 INFO:tasks.workunit.client.0.smithi027.stdout: "up_thru": 0, 2020-09-12T14:04:16.661 INFO:tasks.workunit.client.0.smithi027.stdout: "down_at": 37, 2020-09-12T14:04:16.661 INFO:tasks.workunit.client.0.smithi027.stdout: "lost_at": 0, 2020-09-12T14:04:16.661 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:04:16.661 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:16.661 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.661 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:16.662 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6810", 2020-09-12T14:04:16.662 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49480 2020-09-12T14:04:16.662 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.662 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.662 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:16.662 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6811", 2020-09-12T14:04:16.662 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49480 2020-09-12T14:04:16.663 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.663 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.663 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.663 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addrs": { 2020-09-12T14:04:16.663 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:16.663 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.663 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:16.663 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6812", 2020-09-12T14:04:16.664 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49480 2020-09-12T14:04:16.664 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.664 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.664 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:16.664 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6813", 2020-09-12T14:04:16.664 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49480 2020-09-12T14:04:16.664 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.665 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.665 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.665 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addrs": { 2020-09-12T14:04:16.665 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:16.665 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.665 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:16.665 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6816", 2020-09-12T14:04:16.665 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49480 2020-09-12T14:04:16.666 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.666 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.666 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:16.666 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6817", 2020-09-12T14:04:16.666 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49480 2020-09-12T14:04:16.666 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.666 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.667 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.667 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addrs": { 2020-09-12T14:04:16.667 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:16.667 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.667 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:16.667 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6814", 2020-09-12T14:04:16.667 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49480 2020-09-12T14:04:16.667 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.668 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.668 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:16.668 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6815", 2020-09-12T14:04:16.668 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49480 2020-09-12T14:04:16.668 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.668 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.668 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.668 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:6811/49480", 2020-09-12T14:04:16.669 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addr": "127.0.0.1:6813/49480", 2020-09-12T14:04:16.669 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addr": "127.0.0.1:6817/49480", 2020-09-12T14:04:16.669 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addr": "127.0.0.1:6815/49480", 2020-09-12T14:04:16.669 INFO:tasks.workunit.client.0.smithi027.stdout: "state": [ 2020-09-12T14:04:16.669 INFO:tasks.workunit.client.0.smithi027.stdout: "exists", 2020-09-12T14:04:16.669 INFO:tasks.workunit.client.0.smithi027.stdout: "up" 2020-09-12T14:04:16.669 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.670 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.670 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:16.670 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_xinfo": [ 2020-09-12T14:04:16.670 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.670 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:04:16.670 INFO:tasks.workunit.client.0.smithi027.stdout: "down_stamp": "2020-09-12T14:03:30.043982+0000", 2020-09-12T14:04:16.670 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_probability": 0, 2020-09-12T14:04:16.670 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_interval": 0, 2020-09-12T14:04:16.671 INFO:tasks.workunit.client.0.smithi027.stdout: "features": 4540138292841938943, 2020-09-12T14:04:16.671 INFO:tasks.workunit.client.0.smithi027.stdout: "old_weight": 0, 2020-09-12T14:04:16.671 INFO:tasks.workunit.client.0.smithi027.stdout: "last_purged_snaps_scrub": "2020-09-12T14:02:26.580267+0000", 2020-09-12T14:04:16.671 INFO:tasks.workunit.client.0.smithi027.stdout: "dead_epoch": 0 2020-09-12T14:04:16.671 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.671 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.671 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:04:16.672 INFO:tasks.workunit.client.0.smithi027.stdout: "down_stamp": "2020-09-12T14:03:30.629689+0000", 2020-09-12T14:04:16.672 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_probability": 0, 2020-09-12T14:04:16.672 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_interval": 0, 2020-09-12T14:04:16.672 INFO:tasks.workunit.client.0.smithi027.stdout: "features": 4540138292841938943, 2020-09-12T14:04:16.672 INFO:tasks.workunit.client.0.smithi027.stdout: "old_weight": 0, 2020-09-12T14:04:16.672 INFO:tasks.workunit.client.0.smithi027.stdout: "last_purged_snaps_scrub": "2020-09-12T14:02:34.628567+0000", 2020-09-12T14:04:16.672 INFO:tasks.workunit.client.0.smithi027.stdout: "dead_epoch": 0 2020-09-12T14:04:16.672 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.673 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.673 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:04:16.673 INFO:tasks.workunit.client.0.smithi027.stdout: "down_stamp": "2020-09-12T14:03:31.229169+0000", 2020-09-12T14:04:16.673 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_probability": 0, 2020-09-12T14:04:16.673 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_interval": 0, 2020-09-12T14:04:16.673 INFO:tasks.workunit.client.0.smithi027.stdout: "features": 4540138292841938943, 2020-09-12T14:04:16.673 INFO:tasks.workunit.client.0.smithi027.stdout: "old_weight": 0, 2020-09-12T14:04:16.673 INFO:tasks.workunit.client.0.smithi027.stdout: "last_purged_snaps_scrub": "2020-09-12T14:02:42.728129+0000", 2020-09-12T14:04:16.674 INFO:tasks.workunit.client.0.smithi027.stdout: "dead_epoch": 0 2020-09-12T14:04:16.674 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.674 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:16.674 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_upmap": [], 2020-09-12T14:04:16.674 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_upmap_items": [], 2020-09-12T14:04:16.674 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_temp": [], 2020-09-12T14:04:16.674 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_temp": [], 2020-09-12T14:04:16.674 INFO:tasks.workunit.client.0.smithi027.stdout: "blocklist": {}, 2020-09-12T14:04:16.675 INFO:tasks.workunit.client.0.smithi027.stdout: "erasure_code_profiles": { 2020-09-12T14:04:16.675 INFO:tasks.workunit.client.0.smithi027.stdout: "default": { 2020-09-12T14:04:16.675 INFO:tasks.workunit.client.0.smithi027.stdout: "k": "2", 2020-09-12T14:04:16.675 INFO:tasks.workunit.client.0.smithi027.stdout: "m": "2", 2020-09-12T14:04:16.675 INFO:tasks.workunit.client.0.smithi027.stdout: "plugin": "jerasure", 2020-09-12T14:04:16.675 INFO:tasks.workunit.client.0.smithi027.stdout: "technique": "reed_sol_van" 2020-09-12T14:04:16.675 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.676 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.676 INFO:tasks.workunit.client.0.smithi027.stdout: "removed_snaps_queue": [], 2020-09-12T14:04:16.676 INFO:tasks.workunit.client.0.smithi027.stdout: "new_removed_snaps": [], 2020-09-12T14:04:16.676 INFO:tasks.workunit.client.0.smithi027.stdout: "new_purged_snaps": [], 2020-09-12T14:04:16.676 INFO:tasks.workunit.client.0.smithi027.stdout: "crush_node_flags": {}, 2020-09-12T14:04:16.676 INFO:tasks.workunit.client.0.smithi027.stdout: "device_class_flags": {} 2020-09-12T14:04:16.676 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.676 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_metadata": [ 2020-09-12T14:04:16.677 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.677 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:04:16.677 INFO:tasks.workunit.client.0.smithi027.stdout: "arch": "x86_64", 2020-09-12T14:04:16.677 INFO:tasks.workunit.client.0.smithi027.stdout: "back_addr": "[v2:127.0.0.1:6804/49081,v1:127.0.0.1:6805/49081]", 2020-09-12T14:04:16.677 INFO:tasks.workunit.client.0.smithi027.stdout: "back_iface": "", 2020-09-12T14:04:16.677 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs": "1", 2020-09-12T14:04:16.677 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_db": "0", 2020-09-12T14:04:16.677 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_wal": "0", 2020-09-12T14:04:16.678 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_single_shared_device": "1", 2020-09-12T14:04:16.678 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_access_mode": "file", 2020-09-12T14:04:16.678 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_block_size": "4096", 2020-09-12T14:04:16.678 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_devices": "sda", 2020-09-12T14:04:16.678 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_driver": "KernelDevice", 2020-09-12T14:04:16.678 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2020-09-12T14:04:16.678 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_rotational": "1", 2020-09-12T14:04:16.679 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_size": "107374182400", 2020-09-12T14:04:16.679 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_support_discard": "0", 2020-09-12T14:04:16.679 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_type": "hdd", 2020-09-12T14:04:16.679 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_release": "pacific", 2020-09-12T14:04:16.679 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version": "ceph version 16.0.0-5485-gf346cc73d15 (f346cc73d15a568d3bec1c99a58953de76f57b85) pacific (dev)", 2020-09-12T14:04:16.679 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version_short": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:04:16.679 INFO:tasks.workunit.client.0.smithi027.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2020-09-12T14:04:16.679 INFO:tasks.workunit.client.0.smithi027.stdout: "default_device_class": "hdd", 2020-09-12T14:04:16.680 INFO:tasks.workunit.client.0.smithi027.stdout: "device_ids": "sda=ST1000NM0033-9ZM173_Z1W4H3N7", 2020-09-12T14:04:16.680 INFO:tasks.workunit.client.0.smithi027.stdout: "device_paths": "sda=/dev/disk/by-path/pci-0000:00:1f.2-ata-1", 2020-09-12T14:04:16.680 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": "sda", 2020-09-12T14:04:16.680 INFO:tasks.workunit.client.0.smithi027.stdout: "distro": "centos", 2020-09-12T14:04:16.680 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_description": "CentOS Linux 8 (Core)", 2020-09-12T14:04:16.680 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_version": "8", 2020-09-12T14:04:16.680 INFO:tasks.workunit.client.0.smithi027.stdout: "front_addr": "[v2:127.0.0.1:6802/49081,v1:127.0.0.1:6803/49081]", 2020-09-12T14:04:16.680 INFO:tasks.workunit.client.0.smithi027.stdout: "front_iface": "", 2020-09-12T14:04:16.681 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/49081,v1:127.0.0.1:6809/49081]", 2020-09-12T14:04:16.681 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/49081,v1:127.0.0.1:6807/49081]", 2020-09-12T14:04:16.681 INFO:tasks.workunit.client.0.smithi027.stdout: "hostname": "smithi027", 2020-09-12T14:04:16.681 INFO:tasks.workunit.client.0.smithi027.stdout: "journal_rotational": "1", 2020-09-12T14:04:16.681 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_description": "#1 SMP Sun Jul 26 03:54:29 UTC 2020", 2020-09-12T14:04:16.681 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_version": "4.18.0-193.14.2.el8_2.x86_64", 2020-09-12T14:04:16.681 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_swap_kb": "0", 2020-09-12T14:04:16.682 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_total_kb": "32659336", 2020-09-12T14:04:16.682 INFO:tasks.workunit.client.0.smithi027.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2020-09-12T14:04:16.682 INFO:tasks.workunit.client.0.smithi027.stdout: "objectstore_numa_unknown_devices": "sda", 2020-09-12T14:04:16.682 INFO:tasks.workunit.client.0.smithi027.stdout: "os": "Linux", 2020-09-12T14:04:16.682 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_data": "td/divergent-priors/0", 2020-09-12T14:04:16.682 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_objectstore": "bluestore", 2020-09-12T14:04:16.682 INFO:tasks.workunit.client.0.smithi027.stdout: "osdspec_affinity": "", 2020-09-12T14:04:16.682 INFO:tasks.workunit.client.0.smithi027.stdout: "rotational": "1" 2020-09-12T14:04:16.683 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.683 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.683 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:04:16.683 INFO:tasks.workunit.client.0.smithi027.stdout: "arch": "x86_64", 2020-09-12T14:04:16.683 INFO:tasks.workunit.client.0.smithi027.stdout: "back_addr": "[v2:127.0.0.1:6812/44126,v1:127.0.0.1:6813/44126]", 2020-09-12T14:04:16.683 INFO:tasks.workunit.client.0.smithi027.stdout: "back_iface": "", 2020-09-12T14:04:16.683 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs": "1", 2020-09-12T14:04:16.683 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_db": "0", 2020-09-12T14:04:16.684 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_wal": "0", 2020-09-12T14:04:16.684 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_single_shared_device": "1", 2020-09-12T14:04:16.684 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_access_mode": "file", 2020-09-12T14:04:16.684 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_block_size": "4096", 2020-09-12T14:04:16.684 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_devices": "sda", 2020-09-12T14:04:16.684 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_driver": "KernelDevice", 2020-09-12T14:04:16.684 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2020-09-12T14:04:16.685 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_rotational": "1", 2020-09-12T14:04:16.685 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_size": "107374182400", 2020-09-12T14:04:16.685 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_support_discard": "0", 2020-09-12T14:04:16.685 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_type": "hdd", 2020-09-12T14:04:16.685 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_release": "pacific", 2020-09-12T14:04:16.685 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version": "ceph version 16.0.0-5485-gf346cc73d15 (f346cc73d15a568d3bec1c99a58953de76f57b85) pacific (dev)", 2020-09-12T14:04:16.685 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version_short": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:04:16.685 INFO:tasks.workunit.client.0.smithi027.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2020-09-12T14:04:16.686 INFO:tasks.workunit.client.0.smithi027.stdout: "default_device_class": "hdd", 2020-09-12T14:04:16.686 INFO:tasks.workunit.client.0.smithi027.stdout: "device_ids": "sda=ST1000NM0033-9ZM173_Z1W4H3N7", 2020-09-12T14:04:16.686 INFO:tasks.workunit.client.0.smithi027.stdout: "device_paths": "sda=/dev/disk/by-path/pci-0000:00:1f.2-ata-1", 2020-09-12T14:04:16.686 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": "sda", 2020-09-12T14:04:16.686 INFO:tasks.workunit.client.0.smithi027.stdout: "distro": "centos", 2020-09-12T14:04:16.686 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_description": "CentOS Linux 8 (Core)", 2020-09-12T14:04:16.686 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_version": "8", 2020-09-12T14:04:16.687 INFO:tasks.workunit.client.0.smithi027.stdout: "front_addr": "[v2:127.0.0.1:6810/44126,v1:127.0.0.1:6811/44126]", 2020-09-12T14:04:16.687 INFO:tasks.workunit.client.0.smithi027.stdout: "front_iface": "", 2020-09-12T14:04:16.687 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/44126,v1:127.0.0.1:6817/44126]", 2020-09-12T14:04:16.687 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/44126,v1:127.0.0.1:6815/44126]", 2020-09-12T14:04:16.687 INFO:tasks.workunit.client.0.smithi027.stdout: "hostname": "smithi027", 2020-09-12T14:04:16.687 INFO:tasks.workunit.client.0.smithi027.stdout: "journal_rotational": "1", 2020-09-12T14:04:16.687 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_description": "#1 SMP Sun Jul 26 03:54:29 UTC 2020", 2020-09-12T14:04:16.687 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_version": "4.18.0-193.14.2.el8_2.x86_64", 2020-09-12T14:04:16.688 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_swap_kb": "0", 2020-09-12T14:04:16.688 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_total_kb": "32659336", 2020-09-12T14:04:16.688 INFO:tasks.workunit.client.0.smithi027.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2020-09-12T14:04:16.688 INFO:tasks.workunit.client.0.smithi027.stdout: "objectstore_numa_unknown_devices": "sda", 2020-09-12T14:04:16.688 INFO:tasks.workunit.client.0.smithi027.stdout: "os": "Linux", 2020-09-12T14:04:16.688 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_data": "td/divergent-priors/1", 2020-09-12T14:04:16.688 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_objectstore": "bluestore", 2020-09-12T14:04:16.689 INFO:tasks.workunit.client.0.smithi027.stdout: "osdspec_affinity": "", 2020-09-12T14:04:16.689 INFO:tasks.workunit.client.0.smithi027.stdout: "rotational": "1" 2020-09-12T14:04:16.689 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.689 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.689 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:04:16.689 INFO:tasks.workunit.client.0.smithi027.stdout: "arch": "x86_64", 2020-09-12T14:04:16.689 INFO:tasks.workunit.client.0.smithi027.stdout: "back_addr": "[v2:127.0.0.1:6812/49480,v1:127.0.0.1:6813/49480]", 2020-09-12T14:04:16.690 INFO:tasks.workunit.client.0.smithi027.stdout: "back_iface": "", 2020-09-12T14:04:16.690 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs": "1", 2020-09-12T14:04:16.690 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_db": "0", 2020-09-12T14:04:16.690 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_wal": "0", 2020-09-12T14:04:16.690 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_single_shared_device": "1", 2020-09-12T14:04:16.690 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_access_mode": "file", 2020-09-12T14:04:16.690 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_block_size": "4096", 2020-09-12T14:04:16.690 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_devices": "sda", 2020-09-12T14:04:16.691 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_driver": "KernelDevice", 2020-09-12T14:04:16.691 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2020-09-12T14:04:16.691 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_rotational": "1", 2020-09-12T14:04:16.691 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_size": "107374182400", 2020-09-12T14:04:16.691 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_support_discard": "0", 2020-09-12T14:04:16.691 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_type": "hdd", 2020-09-12T14:04:16.691 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_release": "pacific", 2020-09-12T14:04:16.691 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version": "ceph version 16.0.0-5485-gf346cc73d15 (f346cc73d15a568d3bec1c99a58953de76f57b85) pacific (dev)", 2020-09-12T14:04:16.692 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version_short": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:04:16.692 INFO:tasks.workunit.client.0.smithi027.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2020-09-12T14:04:16.692 INFO:tasks.workunit.client.0.smithi027.stdout: "default_device_class": "hdd", 2020-09-12T14:04:16.692 INFO:tasks.workunit.client.0.smithi027.stdout: "device_ids": "sda=ST1000NM0033-9ZM173_Z1W4H3N7", 2020-09-12T14:04:16.692 INFO:tasks.workunit.client.0.smithi027.stdout: "device_paths": "sda=/dev/disk/by-path/pci-0000:00:1f.2-ata-1", 2020-09-12T14:04:16.692 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": "sda", 2020-09-12T14:04:16.692 INFO:tasks.workunit.client.0.smithi027.stdout: "distro": "centos", 2020-09-12T14:04:16.693 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_description": "CentOS Linux 8 (Core)", 2020-09-12T14:04:16.693 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_version": "8", 2020-09-12T14:04:16.693 INFO:tasks.workunit.client.0.smithi027.stdout: "front_addr": "[v2:127.0.0.1:6810/49480,v1:127.0.0.1:6811/49480]", 2020-09-12T14:04:16.693 INFO:tasks.workunit.client.0.smithi027.stdout: "front_iface": "", 2020-09-12T14:04:16.693 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/49480,v1:127.0.0.1:6817/49480]", 2020-09-12T14:04:16.693 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/49480,v1:127.0.0.1:6815/49480]", 2020-09-12T14:04:16.693 INFO:tasks.workunit.client.0.smithi027.stdout: "hostname": "smithi027", 2020-09-12T14:04:16.693 INFO:tasks.workunit.client.0.smithi027.stdout: "journal_rotational": "1", 2020-09-12T14:04:16.694 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_description": "#1 SMP Sun Jul 26 03:54:29 UTC 2020", 2020-09-12T14:04:16.694 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_version": "4.18.0-193.14.2.el8_2.x86_64", 2020-09-12T14:04:16.694 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_swap_kb": "0", 2020-09-12T14:04:16.694 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_total_kb": "32659336", 2020-09-12T14:04:16.694 INFO:tasks.workunit.client.0.smithi027.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2020-09-12T14:04:16.694 INFO:tasks.workunit.client.0.smithi027.stdout: "objectstore_numa_unknown_devices": "sda", 2020-09-12T14:04:16.694 INFO:tasks.workunit.client.0.smithi027.stdout: "os": "Linux", 2020-09-12T14:04:16.694 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_data": "td/divergent-priors/2", 2020-09-12T14:04:16.695 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_objectstore": "bluestore", 2020-09-12T14:04:16.695 INFO:tasks.workunit.client.0.smithi027.stdout: "osdspec_affinity": "", 2020-09-12T14:04:16.695 INFO:tasks.workunit.client.0.smithi027.stdout: "rotational": "1" 2020-09-12T14:04:16.695 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.695 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:16.695 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap_clean_epochs": { 2020-09-12T14:04:16.695 INFO:tasks.workunit.client.0.smithi027.stdout: "min_last_epoch_clean": 0, 2020-09-12T14:04:16.696 INFO:tasks.workunit.client.0.smithi027.stdout: "last_epoch_clean": { 2020-09-12T14:04:16.696 INFO:tasks.workunit.client.0.smithi027.stdout: "per_pool": [] 2020-09-12T14:04:16.696 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.696 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_epochs": [ 2020-09-12T14:04:16.696 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.696 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:04:16.696 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 40 2020-09-12T14:04:16.696 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.697 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.697 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:04:16.697 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 10 2020-09-12T14:04:16.697 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.697 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.697 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:04:16.697 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 41 2020-09-12T14:04:16.697 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.698 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.698 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.698 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap_first_committed": 1, 2020-09-12T14:04:16.698 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap_last_committed": 47, 2020-09-12T14:04:16.698 INFO:tasks.workunit.client.0.smithi027.stdout: "crushmap": { 2020-09-12T14:04:16.698 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": [ 2020-09-12T14:04:16.698 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.698 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:04:16.699 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd.0", 2020-09-12T14:04:16.699 INFO:tasks.workunit.client.0.smithi027.stdout: "class": "hdd" 2020-09-12T14:04:16.699 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.699 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.699 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:04:16.699 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd.1", 2020-09-12T14:04:16.699 INFO:tasks.workunit.client.0.smithi027.stdout: "class": "hdd" 2020-09-12T14:04:16.700 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.700 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.700 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:04:16.700 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd.2", 2020-09-12T14:04:16.700 INFO:tasks.workunit.client.0.smithi027.stdout: "class": "hdd" 2020-09-12T14:04:16.700 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.700 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:16.700 INFO:tasks.workunit.client.0.smithi027.stdout: "types": [ 2020-09-12T14:04:16.701 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.701 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 0, 2020-09-12T14:04:16.701 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd" 2020-09-12T14:04:16.701 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.701 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.701 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 1, 2020-09-12T14:04:16.701 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "host" 2020-09-12T14:04:16.701 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.702 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.702 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 2, 2020-09-12T14:04:16.702 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "chassis" 2020-09-12T14:04:16.702 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.702 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.702 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 3, 2020-09-12T14:04:16.702 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "rack" 2020-09-12T14:04:16.702 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.703 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.703 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 4, 2020-09-12T14:04:16.703 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "row" 2020-09-12T14:04:16.703 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.703 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.703 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 5, 2020-09-12T14:04:16.703 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "pdu" 2020-09-12T14:04:16.703 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.704 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.704 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 6, 2020-09-12T14:04:16.704 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "pod" 2020-09-12T14:04:16.704 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.704 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.704 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 7, 2020-09-12T14:04:16.704 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "room" 2020-09-12T14:04:16.705 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.705 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.705 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 8, 2020-09-12T14:04:16.705 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "datacenter" 2020-09-12T14:04:16.705 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.705 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.705 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 9, 2020-09-12T14:04:16.705 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "zone" 2020-09-12T14:04:16.706 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.706 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.706 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 10, 2020-09-12T14:04:16.706 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "region" 2020-09-12T14:04:16.706 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.706 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.706 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 11, 2020-09-12T14:04:16.707 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "root" 2020-09-12T14:04:16.707 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.707 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:16.707 INFO:tasks.workunit.client.0.smithi027.stdout: "buckets": [ 2020-09-12T14:04:16.707 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.707 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -1, 2020-09-12T14:04:16.707 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "default", 2020-09-12T14:04:16.707 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 11, 2020-09-12T14:04:16.708 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "root", 2020-09-12T14:04:16.708 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:04:16.708 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:04:16.708 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:04:16.708 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:04:16.708 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.708 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -3, 2020-09-12T14:04:16.708 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:04:16.709 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:04:16.709 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.709 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.709 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.709 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.709 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -2, 2020-09-12T14:04:16.709 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "default~hdd", 2020-09-12T14:04:16.709 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 11, 2020-09-12T14:04:16.710 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "root", 2020-09-12T14:04:16.710 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:04:16.710 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:04:16.710 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:04:16.710 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:04:16.710 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.710 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -4, 2020-09-12T14:04:16.710 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:04:16.711 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:04:16.711 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.711 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.711 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.711 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.711 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -3, 2020-09-12T14:04:16.711 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "smithi027", 2020-09-12T14:04:16.712 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 1, 2020-09-12T14:04:16.712 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "host", 2020-09-12T14:04:16.712 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:04:16.712 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:04:16.712 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:04:16.712 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:04:16.712 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.712 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:04:16.713 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:04:16.713 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:04:16.713 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.713 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.713 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:04:16.713 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:04:16.713 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 1 2020-09-12T14:04:16.713 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.714 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.714 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:04:16.714 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:04:16.714 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 2 2020-09-12T14:04:16.714 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.714 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.714 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.714 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.715 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -4, 2020-09-12T14:04:16.715 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "smithi027~hdd", 2020-09-12T14:04:16.715 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 1, 2020-09-12T14:04:16.715 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "host", 2020-09-12T14:04:16.715 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:04:16.715 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:04:16.715 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:04:16.715 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:04:16.716 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.716 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:04:16.716 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:04:16.716 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:04:16.716 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.716 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.716 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:04:16.717 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:04:16.717 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 1 2020-09-12T14:04:16.717 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.717 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.717 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:04:16.717 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:04:16.717 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 2 2020-09-12T14:04:16.717 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.718 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.718 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.718 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:16.718 INFO:tasks.workunit.client.0.smithi027.stdout: "rules": [ 2020-09-12T14:04:16.718 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.718 INFO:tasks.workunit.client.0.smithi027.stdout: "rule_id": 0, 2020-09-12T14:04:16.718 INFO:tasks.workunit.client.0.smithi027.stdout: "rule_name": "replicated_rule", 2020-09-12T14:04:16.718 INFO:tasks.workunit.client.0.smithi027.stdout: "ruleset": 0, 2020-09-12T14:04:16.719 INFO:tasks.workunit.client.0.smithi027.stdout: "type": 1, 2020-09-12T14:04:16.719 INFO:tasks.workunit.client.0.smithi027.stdout: "min_size": 1, 2020-09-12T14:04:16.719 INFO:tasks.workunit.client.0.smithi027.stdout: "max_size": 10, 2020-09-12T14:04:16.719 INFO:tasks.workunit.client.0.smithi027.stdout: "steps": [ 2020-09-12T14:04:16.719 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.719 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "take", 2020-09-12T14:04:16.719 INFO:tasks.workunit.client.0.smithi027.stdout: "item": -1, 2020-09-12T14:04:16.720 INFO:tasks.workunit.client.0.smithi027.stdout: "item_name": "default" 2020-09-12T14:04:16.720 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.720 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.720 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "choose_firstn", 2020-09-12T14:04:16.720 INFO:tasks.workunit.client.0.smithi027.stdout: "num": 0, 2020-09-12T14:04:16.720 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "osd" 2020-09-12T14:04:16.720 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.720 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.721 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "emit" 2020-09-12T14:04:16.721 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.721 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:16.721 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.721 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:16.721 INFO:tasks.workunit.client.0.smithi027.stdout: "tunables": { 2020-09-12T14:04:16.721 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_local_tries": 0, 2020-09-12T14:04:16.721 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_local_fallback_tries": 0, 2020-09-12T14:04:16.722 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_total_tries": 50, 2020-09-12T14:04:16.722 INFO:tasks.workunit.client.0.smithi027.stdout: "chooseleaf_descend_once": 1, 2020-09-12T14:04:16.722 INFO:tasks.workunit.client.0.smithi027.stdout: "chooseleaf_vary_r": 1, 2020-09-12T14:04:16.722 INFO:tasks.workunit.client.0.smithi027.stdout: "chooseleaf_stable": 1, 2020-09-12T14:04:16.722 INFO:tasks.workunit.client.0.smithi027.stdout: "straw_calc_version": 1, 2020-09-12T14:04:16.722 INFO:tasks.workunit.client.0.smithi027.stdout: "allowed_bucket_algs": 54, 2020-09-12T14:04:16.722 INFO:tasks.workunit.client.0.smithi027.stdout: "profile": "jewel", 2020-09-12T14:04:16.723 INFO:tasks.workunit.client.0.smithi027.stdout: "optimal_tunables": 1, 2020-09-12T14:04:16.723 INFO:tasks.workunit.client.0.smithi027.stdout: "legacy_tunables": 0, 2020-09-12T14:04:16.723 INFO:tasks.workunit.client.0.smithi027.stdout: "minimum_required_version": "jewel", 2020-09-12T14:04:16.723 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables": 1, 2020-09-12T14:04:16.723 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables2": 1, 2020-09-12T14:04:16.723 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v2_rules": 0, 2020-09-12T14:04:16.723 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables3": 1, 2020-09-12T14:04:16.723 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v3_rules": 0, 2020-09-12T14:04:16.724 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v4_buckets": 1, 2020-09-12T14:04:16.724 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables5": 1, 2020-09-12T14:04:16.724 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v5_rules": 0 2020-09-12T14:04:16.724 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.724 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_args": {} 2020-09-12T14:04:16.724 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.724 INFO:tasks.workunit.client.0.smithi027.stdout: "fsmap": { 2020-09-12T14:04:16.724 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 1, 2020-09-12T14:04:16.725 INFO:tasks.workunit.client.0.smithi027.stdout: "default_fscid": -1, 2020-09-12T14:04:16.725 INFO:tasks.workunit.client.0.smithi027.stdout: "compat": { 2020-09-12T14:04:16.725 INFO:tasks.workunit.client.0.smithi027.stdout: "compat": {}, 2020-09-12T14:04:16.725 INFO:tasks.workunit.client.0.smithi027.stdout: "ro_compat": {}, 2020-09-12T14:04:16.725 INFO:tasks.workunit.client.0.smithi027.stdout: "incompat": { 2020-09-12T14:04:16.725 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_1": "base v0.20", 2020-09-12T14:04:16.725 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_2": "client writeable ranges", 2020-09-12T14:04:16.726 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_3": "default file layouts on dirs", 2020-09-12T14:04:16.726 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_4": "dir inode in separate object", 2020-09-12T14:04:16.726 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_5": "mds uses versioned encoding", 2020-09-12T14:04:16.726 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_6": "dirfrag is stored in omap", 2020-09-12T14:04:16.726 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_8": "no anchor table", 2020-09-12T14:04:16.726 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_9": "file layout v2", 2020-09-12T14:04:16.726 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_10": "snaprealm v2" 2020-09-12T14:04:16.726 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.727 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.727 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_flags": { 2020-09-12T14:04:16.727 INFO:tasks.workunit.client.0.smithi027.stdout: "enable_multiple": false, 2020-09-12T14:04:16.727 INFO:tasks.workunit.client.0.smithi027.stdout: "ever_enabled_multiple": false 2020-09-12T14:04:16.727 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.727 INFO:tasks.workunit.client.0.smithi027.stdout: "standbys": [], 2020-09-12T14:04:16.727 INFO:tasks.workunit.client.0.smithi027.stdout: "filesystems": [] 2020-09-12T14:04:16.727 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.728 INFO:tasks.workunit.client.0.smithi027.stdout: "mdsmap_first_committed": 1, 2020-09-12T14:04:16.728 INFO:tasks.workunit.client.0.smithi027.stdout: "mdsmap_last_committed": 1, 2020-09-12T14:04:16.728 INFO:tasks.workunit.client.0.smithi027.stdout: "auth": { 2020-09-12T14:04:16.728 INFO:tasks.workunit.client.0.smithi027.stdout: "first_committed": 1, 2020-09-12T14:04:16.728 INFO:tasks.workunit.client.0.smithi027.stdout: "last_committed": 5, 2020-09-12T14:04:16.728 INFO:tasks.workunit.client.0.smithi027.stdout: "num_secrets": 3 2020-09-12T14:04:16.728 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.729 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg": 32, 2020-09-12T14:04:16.729 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_active": 32, 2020-09-12T14:04:16.729 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_unknown": 0, 2020-09-12T14:04:16.729 INFO:tasks.workunit.client.0.smithi027.stdout: "num_osd": 3, 2020-09-12T14:04:16.729 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_sum": { 2020-09-12T14:04:16.729 INFO:tasks.workunit.client.0.smithi027.stdout: "stat_sum": { 2020-09-12T14:04:16.729 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes": 48300, 2020-09-12T14:04:16.729 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects": 100, 2020-09-12T14:04:16.730 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_clones": 0, 2020-09-12T14:04:16.730 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_copies": 300, 2020-09-12T14:04:16.730 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing_on_primary": 0, 2020-09-12T14:04:16.730 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing": 0, 2020-09-12T14:04:16.730 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_degraded": 100, 2020-09-12T14:04:16.730 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_misplaced": 0, 2020-09-12T14:04:16.730 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_unfound": 0, 2020-09-12T14:04:16.730 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_dirty": 100, 2020-09-12T14:04:16.731 INFO:tasks.workunit.client.0.smithi027.stdout: "num_whiteouts": 0, 2020-09-12T14:04:16.731 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read": 0, 2020-09-12T14:04:16.731 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read_kb": 0, 2020-09-12T14:04:16.731 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write": 100, 2020-09-12T14:04:16.731 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write_kb": 100, 2020-09-12T14:04:16.731 INFO:tasks.workunit.client.0.smithi027.stdout: "num_scrub_errors": 0, 2020-09-12T14:04:16.731 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shallow_scrub_errors": 0, 2020-09-12T14:04:16.731 INFO:tasks.workunit.client.0.smithi027.stdout: "num_deep_scrub_errors": 0, 2020-09-12T14:04:16.732 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_recovered": 0, 2020-09-12T14:04:16.732 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_recovered": 0, 2020-09-12T14:04:16.732 INFO:tasks.workunit.client.0.smithi027.stdout: "num_keys_recovered": 0, 2020-09-12T14:04:16.732 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_omap": 0, 2020-09-12T14:04:16.732 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_hit_set_archive": 0, 2020-09-12T14:04:16.732 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_hit_set_archive": 0, 2020-09-12T14:04:16.732 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush": 0, 2020-09-12T14:04:16.733 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_kb": 0, 2020-09-12T14:04:16.733 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict": 0, 2020-09-12T14:04:16.733 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_kb": 0, 2020-09-12T14:04:16.733 INFO:tasks.workunit.client.0.smithi027.stdout: "num_promote": 0, 2020-09-12T14:04:16.733 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_high": 0, 2020-09-12T14:04:16.733 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_low": 0, 2020-09-12T14:04:16.733 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_some": 0, 2020-09-12T14:04:16.733 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_full": 0, 2020-09-12T14:04:16.734 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_pinned": 0, 2020-09-12T14:04:16.734 INFO:tasks.workunit.client.0.smithi027.stdout: "num_legacy_snapsets": 0, 2020-09-12T14:04:16.734 INFO:tasks.workunit.client.0.smithi027.stdout: "num_large_omap_objects": 0, 2020-09-12T14:04:16.734 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_manifest": 0, 2020-09-12T14:04:16.734 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_bytes": 0, 2020-09-12T14:04:16.734 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_keys": 0, 2020-09-12T14:04:16.734 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_repaired": 0 2020-09-12T14:04:16.734 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.735 INFO:tasks.workunit.client.0.smithi027.stdout: "store_stats": { 2020-09-12T14:04:16.735 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 0, 2020-09-12T14:04:16.735 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 0, 2020-09-12T14:04:16.735 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:04:16.735 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 0, 2020-09-12T14:04:16.735 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 0, 2020-09-12T14:04:16.735 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:04:16.735 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:04:16.736 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:04:16.736 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 0, 2020-09-12T14:04:16.736 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 0 2020-09-12T14:04:16.736 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.736 INFO:tasks.workunit.client.0.smithi027.stdout: "log_size": 512, 2020-09-12T14:04:16.736 INFO:tasks.workunit.client.0.smithi027.stdout: "ondisk_log_size": 512, 2020-09-12T14:04:16.736 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 64, 2020-09-12T14:04:16.736 INFO:tasks.workunit.client.0.smithi027.stdout: "acting": 64, 2020-09-12T14:04:16.737 INFO:tasks.workunit.client.0.smithi027.stdout: "num_store_stats": 0 2020-09-12T14:04:16.737 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.737 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_sum": { 2020-09-12T14:04:16.737 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 0, 2020-09-12T14:04:16.737 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 0, 2020-09-12T14:04:16.737 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pgs": 65, 2020-09-12T14:04:16.737 INFO:tasks.workunit.client.0.smithi027.stdout: "num_osds": 3, 2020-09-12T14:04:16.738 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_osds": 3, 2020-09-12T14:04:16.738 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_omap_osds": 3, 2020-09-12T14:04:16.738 INFO:tasks.workunit.client.0.smithi027.stdout: "kb": 314572800, 2020-09-12T14:04:16.738 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used": 19596, 2020-09-12T14:04:16.738 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_data": 2220, 2020-09-12T14:04:16.738 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_omap": 1, 2020-09-12T14:04:16.738 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_meta": 17342, 2020-09-12T14:04:16.738 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_avail": 314553204, 2020-09-12T14:04:16.739 INFO:tasks.workunit.client.0.smithi027.stdout: "statfs": { 2020-09-12T14:04:16.739 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 322122547200, 2020-09-12T14:04:16.739 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 322102480896, 2020-09-12T14:04:16.739 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:04:16.739 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 2273280, 2020-09-12T14:04:16.739 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 465574, 2020-09-12T14:04:16.739 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:04:16.739 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:04:16.740 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:04:16.740 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 1928, 2020-09-12T14:04:16.740 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 17758328 2020-09-12T14:04:16.740 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.740 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_peers": [], 2020-09-12T14:04:16.740 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_trim_queue_len": 0, 2020-09-12T14:04:16.740 INFO:tasks.workunit.client.0.smithi027.stdout: "num_snap_trimming": 0, 2020-09-12T14:04:16.740 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shards_repaired": 0, 2020-09-12T14:04:16.741 INFO:tasks.workunit.client.0.smithi027.stdout: "op_queue_age_hist": { 2020-09-12T14:04:16.741 INFO:tasks.workunit.client.0.smithi027.stdout: "histogram": [], 2020-09-12T14:04:16.741 INFO:tasks.workunit.client.0.smithi027.stdout: "upper_bound": 1 2020-09-12T14:04:16.741 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.741 INFO:tasks.workunit.client.0.smithi027.stdout: "perf_stat": { 2020-09-12T14:04:16.741 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ms": 43, 2020-09-12T14:04:16.741 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ms": 43, 2020-09-12T14:04:16.742 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ns": 43000000, 2020-09-12T14:04:16.742 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ns": 43000000 2020-09-12T14:04:16.742 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.742 INFO:tasks.workunit.client.0.smithi027.stdout: "alerts": [], 2020-09-12T14:04:16.742 INFO:tasks.workunit.client.0.smithi027.stdout: "network_ping_times": [] 2020-09-12T14:04:16.742 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.742 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_sum_by_class": { 2020-09-12T14:04:16.742 INFO:tasks.workunit.client.0.smithi027.stdout: "hdd": { 2020-09-12T14:04:16.743 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 0, 2020-09-12T14:04:16.743 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 0, 2020-09-12T14:04:16.743 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pgs": 65, 2020-09-12T14:04:16.743 INFO:tasks.workunit.client.0.smithi027.stdout: "num_osds": 3, 2020-09-12T14:04:16.743 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_osds": 3, 2020-09-12T14:04:16.743 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_omap_osds": 3, 2020-09-12T14:04:16.743 INFO:tasks.workunit.client.0.smithi027.stdout: "kb": 314572800, 2020-09-12T14:04:16.743 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used": 19596, 2020-09-12T14:04:16.744 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_data": 2220, 2020-09-12T14:04:16.744 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_omap": 1, 2020-09-12T14:04:16.744 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_meta": 17342, 2020-09-12T14:04:16.744 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_avail": 314553204, 2020-09-12T14:04:16.744 INFO:tasks.workunit.client.0.smithi027.stdout: "statfs": { 2020-09-12T14:04:16.744 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 322122547200, 2020-09-12T14:04:16.744 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 322102480896, 2020-09-12T14:04:16.744 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:04:16.745 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 2273280, 2020-09-12T14:04:16.745 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 465574, 2020-09-12T14:04:16.745 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:04:16.745 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:04:16.745 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:04:16.745 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 1928, 2020-09-12T14:04:16.745 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 17758328 2020-09-12T14:04:16.745 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.746 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_peers": [], 2020-09-12T14:04:16.746 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_trim_queue_len": 0, 2020-09-12T14:04:16.746 INFO:tasks.workunit.client.0.smithi027.stdout: "num_snap_trimming": 0, 2020-09-12T14:04:16.746 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shards_repaired": 0, 2020-09-12T14:04:16.746 INFO:tasks.workunit.client.0.smithi027.stdout: "op_queue_age_hist": { 2020-09-12T14:04:16.746 INFO:tasks.workunit.client.0.smithi027.stdout: "histogram": [], 2020-09-12T14:04:16.746 INFO:tasks.workunit.client.0.smithi027.stdout: "upper_bound": 1 2020-09-12T14:04:16.747 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.747 INFO:tasks.workunit.client.0.smithi027.stdout: "perf_stat": { 2020-09-12T14:04:16.747 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ms": 43, 2020-09-12T14:04:16.747 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ms": 43, 2020-09-12T14:04:16.747 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ns": 43000000, 2020-09-12T14:04:16.747 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ns": 43000000 2020-09-12T14:04:16.747 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.747 INFO:tasks.workunit.client.0.smithi027.stdout: "alerts": [], 2020-09-12T14:04:16.748 INFO:tasks.workunit.client.0.smithi027.stdout: "network_ping_times": [] 2020-09-12T14:04:16.748 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.748 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.748 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_stats": [ 2020-09-12T14:04:16.748 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.748 INFO:tasks.workunit.client.0.smithi027.stdout: "poolid": 1, 2020-09-12T14:04:16.748 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg": 32, 2020-09-12T14:04:16.748 INFO:tasks.workunit.client.0.smithi027.stdout: "stat_sum": { 2020-09-12T14:04:16.749 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes": 48300, 2020-09-12T14:04:16.749 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects": 100, 2020-09-12T14:04:16.749 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_clones": 0, 2020-09-12T14:04:16.749 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_copies": 300, 2020-09-12T14:04:16.749 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing_on_primary": 0, 2020-09-12T14:04:16.749 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing": 0, 2020-09-12T14:04:16.749 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_degraded": 100, 2020-09-12T14:04:16.749 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_misplaced": 0, 2020-09-12T14:04:16.750 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_unfound": 0, 2020-09-12T14:04:16.750 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_dirty": 100, 2020-09-12T14:04:16.750 INFO:tasks.workunit.client.0.smithi027.stdout: "num_whiteouts": 0, 2020-09-12T14:04:16.750 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read": 0, 2020-09-12T14:04:16.750 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read_kb": 0, 2020-09-12T14:04:16.750 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write": 100, 2020-09-12T14:04:16.750 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write_kb": 100, 2020-09-12T14:04:16.751 INFO:tasks.workunit.client.0.smithi027.stdout: "num_scrub_errors": 0, 2020-09-12T14:04:16.751 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shallow_scrub_errors": 0, 2020-09-12T14:04:16.751 INFO:tasks.workunit.client.0.smithi027.stdout: "num_deep_scrub_errors": 0, 2020-09-12T14:04:16.751 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_recovered": 0, 2020-09-12T14:04:16.751 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_recovered": 0, 2020-09-12T14:04:16.751 INFO:tasks.workunit.client.0.smithi027.stdout: "num_keys_recovered": 0, 2020-09-12T14:04:16.751 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_omap": 0, 2020-09-12T14:04:16.751 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_hit_set_archive": 0, 2020-09-12T14:04:16.752 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_hit_set_archive": 0, 2020-09-12T14:04:16.752 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush": 0, 2020-09-12T14:04:16.752 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_kb": 0, 2020-09-12T14:04:16.752 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict": 0, 2020-09-12T14:04:16.752 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_kb": 0, 2020-09-12T14:04:16.753 INFO:tasks.workunit.client.0.smithi027.stdout: "num_promote": 0, 2020-09-12T14:04:16.753 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_high": 0, 2020-09-12T14:04:16.753 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_low": 0, 2020-09-12T14:04:16.753 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_some": 0, 2020-09-12T14:04:16.753 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_full": 0, 2020-09-12T14:04:16.753 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_pinned": 0, 2020-09-12T14:04:16.753 INFO:tasks.workunit.client.0.smithi027.stdout: "num_legacy_snapsets": 0, 2020-09-12T14:04:16.754 INFO:tasks.workunit.client.0.smithi027.stdout: "num_large_omap_objects": 0, 2020-09-12T14:04:16.754 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_manifest": 0, 2020-09-12T14:04:16.754 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_bytes": 0, 2020-09-12T14:04:16.754 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_keys": 0, 2020-09-12T14:04:16.754 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_repaired": 0 2020-09-12T14:04:16.754 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.754 INFO:tasks.workunit.client.0.smithi027.stdout: "store_stats": { 2020-09-12T14:04:16.754 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 0, 2020-09-12T14:04:16.755 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 0, 2020-09-12T14:04:16.755 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:04:16.755 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 1220608, 2020-09-12T14:04:16.755 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 143228, 2020-09-12T14:04:16.755 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:04:16.755 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:04:16.755 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:04:16.755 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 0, 2020-09-12T14:04:16.756 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 0 2020-09-12T14:04:16.756 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.756 INFO:tasks.workunit.client.0.smithi027.stdout: "log_size": 512, 2020-09-12T14:04:16.756 INFO:tasks.workunit.client.0.smithi027.stdout: "ondisk_log_size": 512, 2020-09-12T14:04:16.756 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 64, 2020-09-12T14:04:16.756 INFO:tasks.workunit.client.0.smithi027.stdout: "acting": 64, 2020-09-12T14:04:16.756 INFO:tasks.workunit.client.0.smithi027.stdout: "num_store_stats": 3 2020-09-12T14:04:16.757 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.757 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:16.757 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_stats": [ 2020-09-12T14:04:16.757 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.757 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:04:16.757 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 171798691850 2020-09-12T14:04:16.757 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.757 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.758 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:04:16.758 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 42949672977 2020-09-12T14:04:16.758 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.758 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.758 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:04:16.758 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 176093659146 2020-09-12T14:04:16.758 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.758 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:16.759 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_by_state": [ 2020-09-12T14:04:16.759 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.759 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "active+undersized+degraded", 2020-09-12T14:04:16.759 INFO:tasks.workunit.client.0.smithi027.stdout: "num": 32 2020-09-12T14:04:16.759 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.759 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:16.759 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_by_osd": [ 2020-09-12T14:04:16.760 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.760 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:04:16.760 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 0, 2020-09-12T14:04:16.760 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 32, 2020-09-12T14:04:16.760 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T14:04:16.760 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.760 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.760 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:04:16.761 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 0, 2020-09-12T14:04:16.761 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 0, 2020-09-12T14:04:16.761 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T14:04:16.761 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.761 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.761 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:04:16.761 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 32, 2020-09-12T14:04:16.761 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 32, 2020-09-12T14:04:16.762 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T14:04:16.762 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.762 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:16.762 INFO:tasks.workunit.client.0.smithi027.stdout: "purged_snaps": [ 2020-09-12T14:04:16.762 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:16.762 INFO:tasks.workunit.client.0.smithi027.stdout: "pool": 1, 2020-09-12T14:04:16.762 INFO:tasks.workunit.client.0.smithi027.stdout: "purged_snaps": {} 2020-09-12T14:04:16.763 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.763 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:16.763 INFO:tasks.workunit.client.0.smithi027.stdout: "servicemap": { 2020-09-12T14:04:16.763 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 4, 2020-09-12T14:04:16.763 INFO:tasks.workunit.client.0.smithi027.stdout: "modified": "2020-09-12T14:03:36.638872+0000", 2020-09-12T14:04:16.763 INFO:tasks.workunit.client.0.smithi027.stdout: "services": { 2020-09-12T14:04:16.763 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": { 2020-09-12T14:04:16.763 INFO:tasks.workunit.client.0.smithi027.stdout: "daemons": { 2020-09-12T14:04:16.764 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": "", 2020-09-12T14:04:16.764 INFO:tasks.workunit.client.0.smithi027.stdout: "0": { 2020-09-12T14:04:16.764 INFO:tasks.workunit.client.0.smithi027.stdout: "start_epoch": 0, 2020-09-12T14:04:16.764 INFO:tasks.workunit.client.0.smithi027.stdout: "start_stamp": "0.000000", 2020-09-12T14:04:16.764 INFO:tasks.workunit.client.0.smithi027.stdout: "gid": 0, 2020-09-12T14:04:16.764 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "(unrecognized address family 0)/0", 2020-09-12T14:04:16.764 INFO:tasks.workunit.client.0.smithi027.stdout: "metadata": {}, 2020-09-12T14:04:16.764 INFO:tasks.workunit.client.0.smithi027.stdout: "task_status": {} 2020-09-12T14:04:16.765 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.765 INFO:tasks.workunit.client.0.smithi027.stdout: "2": { 2020-09-12T14:04:16.765 INFO:tasks.workunit.client.0.smithi027.stdout: "start_epoch": 0, 2020-09-12T14:04:16.765 INFO:tasks.workunit.client.0.smithi027.stdout: "start_stamp": "0.000000", 2020-09-12T14:04:16.765 INFO:tasks.workunit.client.0.smithi027.stdout: "gid": 0, 2020-09-12T14:04:16.765 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "(unrecognized address family 0)/0", 2020-09-12T14:04:16.765 INFO:tasks.workunit.client.0.smithi027.stdout: "metadata": {}, 2020-09-12T14:04:16.765 INFO:tasks.workunit.client.0.smithi027.stdout: "task_status": {} 2020-09-12T14:04:16.766 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.766 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.766 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.766 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.766 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:16.766 INFO:tasks.workunit.client.0.smithi027.stdout: "paxos": { 2020-09-12T14:04:16.766 INFO:tasks.workunit.client.0.smithi027.stdout: "first_committed": 1, 2020-09-12T14:04:16.766 INFO:tasks.workunit.client.0.smithi027.stdout: "last_committed": 372, 2020-09-12T14:04:16.767 INFO:tasks.workunit.client.0.smithi027.stdout: "last_pn": 0, 2020-09-12T14:04:16.767 INFO:tasks.workunit.client.0.smithi027.stdout: "accepted_pn": 0 2020-09-12T14:04:16.767 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:16.767 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:04:16.768 INFO:tasks.workunit.client.0.smithi027.stderr:report 2464296219 2020-09-12T14:04:16.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1604: wait_for_clean: return 1 2020-09-12T14:04:16.768 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:738: TEST_divergent_3: expr 2 + 2 2020-09-12T14:04:16.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:738: TEST_divergent_3: objname=existing_4 2020-09-12T14:04:16.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:739: TEST_divergent_3: echo 'writing non-divergent object existing_4' 2020-09-12T14:04:16.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:740: TEST_divergent_3: ceph pg dump pgs 2020-09-12T14:04:16.769 INFO:tasks.workunit.client.0.smithi027.stdout:writing non-divergent object existing_4 2020-09-12T14:04:16.978 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:04:16.978 INFO:tasks.workunit.client.0.smithi027.stdout:1.1f 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958633+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.978 INFO:tasks.workunit.client.0.smithi027.stdout:1.1e 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958019+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.978 INFO:tasks.workunit.client.0.smithi027.stdout:1.1d 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958447+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.978 INFO:tasks.workunit.client.0.smithi027.stdout:1.1c 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958753+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.979 INFO:tasks.workunit.client.0.smithi027.stdout:1.1b 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958199+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.979 INFO:tasks.workunit.client.0.smithi027.stdout:1.1a 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955975+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.979 INFO:tasks.workunit.client.0.smithi027.stdout:1.19 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.813511+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.979 INFO:tasks.workunit.client.0.smithi027.stdout:1.18 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958032+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.979 INFO:tasks.workunit.client.0.smithi027.stdout:1.17 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958327+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.979 INFO:tasks.workunit.client.0.smithi027.stdout:1.16 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957794+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.979 INFO:tasks.workunit.client.0.smithi027.stdout:1.15 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957055+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.980 INFO:tasks.workunit.client.0.smithi027.stdout:1.14 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955987+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.980 INFO:tasks.workunit.client.0.smithi027.stdout:1.13 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957641+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.980 INFO:tasks.workunit.client.0.smithi027.stdout:1.12 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957558+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.980 INFO:tasks.workunit.client.0.smithi027.stdout:1.11 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957392+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.980 INFO:tasks.workunit.client.0.smithi027.stdout:1.10 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957442+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.980 INFO:tasks.workunit.client.0.smithi027.stdout:1.5 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957006+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.980 INFO:tasks.workunit.client.0.smithi027.stdout:1.4 4 0 4 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957032+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.980 INFO:tasks.workunit.client.0.smithi027.stdout:1.3 4 0 4 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.956677+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.981 INFO:tasks.workunit.client.0.smithi027.stdout:1.2 4 0 4 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.956981+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.981 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958434+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.981 INFO:tasks.workunit.client.0.smithi027.stdout:1.1 4 0 4 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957840+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.981 INFO:tasks.workunit.client.0.smithi027.stdout:1.6 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958222+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.981 INFO:tasks.workunit.client.0.smithi027.stdout:1.7 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955938+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.981 INFO:tasks.workunit.client.0.smithi027.stdout:1.8 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955987+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.981 INFO:tasks.workunit.client.0.smithi027.stdout:1.9 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955983+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.982 INFO:tasks.workunit.client.0.smithi027.stdout:1.a 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958911+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.982 INFO:tasks.workunit.client.0.smithi027.stdout:1.b 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.956643+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.982 INFO:tasks.workunit.client.0.smithi027.stdout:1.c 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.956672+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.982 INFO:tasks.workunit.client.0.smithi027.stdout:1.d 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957160+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.982 INFO:tasks.workunit.client.0.smithi027.stdout:1.e 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.956415+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.982 INFO:tasks.workunit.client.0.smithi027.stdout:1.f 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957591+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:16.982 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:04:16.983 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:04:16.983 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:04:16.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:741: TEST_divergent_3: rados -p test put existing_4 /etc/resolv.conf 2020-09-12T14:04:17.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:743: TEST_divergent_3: WAIT_FOR_CLEAN_TIMEOUT=20 2020-09-12T14:04:17.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:743: TEST_divergent_3: wait_for_clean 2020-09-12T14:04:17.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:04:17.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:04:17.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:04:17.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:04:17.171 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 20 .1 2020-09-12T14:04:17.171 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:04:17.171 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:04:17.171 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:04:17.172 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:04:17.172 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:04:17.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:04:17.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:04:17.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:04:17.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:04:17.229 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:04:17.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:04:17.638 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:04:17.638 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T14:04:17.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:04:17.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:04:17.639 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:04:17.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=171798691853 2020-09-12T14:04:17.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 171798691853 2020-09-12T14:04:17.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-171798691853' 2020-09-12T14:04:17.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:04:17.802 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:04:17.909 INFO:tasks.workunit.client.0.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-09-12T14:04:17.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq= 2020-09-12T14:04:17.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z '' 2020-09-12T14:04:17.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2075: flush_pg_stats: continue 2020-09-12T14:04:17.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:04:17.911 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:04:18.093 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=176093659149 2020-09-12T14:04:18.093 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 176093659149 2020-09-12T14:04:18.093 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-171798691853 2-176093659149' 2020-09-12T14:04:18.093 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:04:18.094 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-171798691853 2020-09-12T14:04:18.094 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:04:18.094 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:04:18.095 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-171798691853 2020-09-12T14:04:18.095 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:04:18.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=171798691853 2020-09-12T14:04:18.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 171798691853' 2020-09-12T14:04:18.096 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 171798691853 2020-09-12T14:04:18.096 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:04:18.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 171798691851 -lt 171798691853 2020-09-12T14:04:18.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:04:19.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:04:19.501 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:04:19.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 171798691853 -lt 171798691853 2020-09-12T14:04:19.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:04:19.920 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-176093659149 2020-09-12T14:04:19.920 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:04:19.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:04:19.921 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-176093659149 2020-09-12T14:04:19.921 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:04:19.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=176093659149 2020-09-12T14:04:19.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 176093659149' 2020-09-12T14:04:19.922 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 176093659149 2020-09-12T14:04:19.922 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:04:20.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 176093659149 -lt 176093659149 2020-09-12T14:04:20.328 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:04:20.329 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:04:20.329 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:04:20.862 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 32 == 0 2020-09-12T14:04:20.862 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:04:20.863 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:04:20.863 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:04:20.863 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:04:20.863 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:04:20.863 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:04:20.864 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:04:21.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:04:21.272 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:04:21.273 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:04:21.273 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:04:21.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 32 2020-09-12T14:04:21.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' -1 2020-09-12T14:04:21.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1598: wait_for_clean: loop=0 2020-09-12T14:04:21.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1599: wait_for_clean: num_active_clean=0 2020-09-12T14:04:21.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:04:21.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:04:21.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:04:21.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:04:21.911 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:04:21.911 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:04:21.911 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:04:21.911 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:04:21.911 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:04:21.912 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:04:22.342 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:04:22.342 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:04:22.343 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:04:22.343 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:04:22.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 32 2020-09-12T14:04:22.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:04:22.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:04:22.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:04:22.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:04:22.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:04:22.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:04:22.881 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:04:22.881 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:04:23.418 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:04:23.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:04:23.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 1 >= 8 )) 2020-09-12T14:04:23.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:04:23.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.2 2020-09-12T14:04:23.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:04:23.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:04:23.620 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:04:23.620 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:04:23.620 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:04:23.621 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:04:23.621 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:04:23.621 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:04:24.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:04:24.029 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:04:24.029 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:04:24.029 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:04:24.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 32 2020-09-12T14:04:24.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:04:24.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:04:24.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:04:24.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:04:24.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:04:24.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:04:24.572 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:04:24.572 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:04:25.106 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:04:25.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:04:25.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 2 >= 8 )) 2020-09-12T14:04:25.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:04:25.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.4 2020-09-12T14:04:25.508 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:04:25.508 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:04:25.508 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:04:25.508 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:04:25.508 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:04:25.509 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:04:25.509 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:04:25.509 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:04:25.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:04:25.912 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:04:25.912 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:04:25.913 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:04:26.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 32 2020-09-12T14:04:26.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:04:26.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:04:26.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:04:26.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:04:26.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:04:26.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:04:26.473 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:04:26.473 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:04:27.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:04:27.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:04:27.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 3 >= 8 )) 2020-09-12T14:04:27.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:04:27.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.8 2020-09-12T14:04:27.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:04:27.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:04:27.813 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:04:27.813 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:04:27.813 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:04:27.814 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:04:27.814 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:04:27.814 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:04:28.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:04:28.232 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:04:28.233 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:04:28.233 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:04:28.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 32 2020-09-12T14:04:28.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:04:28.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:04:28.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:04:28.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:04:28.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:04:28.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:04:28.777 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:04:28.777 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:04:29.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:04:29.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:04:29.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 4 >= 8 )) 2020-09-12T14:04:29.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:04:29.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 1.6 2020-09-12T14:04:30.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:04:30.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:04:30.912 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:04:30.913 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:04:30.913 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:04:30.913 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:04:30.913 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:04:30.913 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:04:31.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:04:31.350 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:04:31.350 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:04:31.351 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:04:31.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 32 2020-09-12T14:04:31.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:04:31.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:04:31.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:04:31.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:04:31.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:04:31.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:04:31.886 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:04:31.886 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:04:32.477 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:04:32.477 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:04:32.477 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 5 >= 8 )) 2020-09-12T14:04:32.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:04:32.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 3.2 2020-09-12T14:04:35.678 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:04:35.678 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:04:35.679 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:04:35.679 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:04:35.679 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:04:35.680 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:04:35.680 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:04:35.680 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:04:36.091 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:04:36.092 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:04:36.092 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:04:36.093 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:04:36.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 32 2020-09-12T14:04:36.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:04:36.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:04:36.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:04:36.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:04:36.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:04:36.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:04:36.624 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:04:36.624 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:04:37.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:04:37.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:04:37.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 6 >= 8 )) 2020-09-12T14:04:37.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:04:37.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 6.4 2020-09-12T14:04:43.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:04:43.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:04:43.569 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:04:43.569 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:04:43.570 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:04:43.570 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:04:43.570 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:04:43.570 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:04:43.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:04:43.982 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:04:43.982 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:04:43.982 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:04:44.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 32 2020-09-12T14:04:44.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:04:44.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:04:44.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:04:44.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:04:44.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:04:44.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:04:44.514 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:04:44.514 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:04:45.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:04:45.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:04:45.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 7 >= 8 )) 2020-09-12T14:04:45.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:04:45.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 7.3 2020-09-12T14:04:52.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:04:52.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:04:52.344 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:04:52.344 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:04:52.344 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:04:52.344 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:04:52.344 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:04:52.344 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:04:52.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:04:52.764 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:04:52.765 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:04:52.765 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:04:53.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 32 2020-09-12T14:04:53.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:04:53.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:04:53.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:04:53.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:04:53.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:04:53.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:04:53.307 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:04:53.307 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:04:53.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:04:53.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:04:53.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 8 >= 8 )) 2020-09-12T14:04:53.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: wait_for_clean: ceph report 2020-09-12T14:04:54.360 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:04:54.360 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_fingerprint": "62ae0e86-0762-4c2d-b8c6-c318affebcfc", 2020-09-12T14:04:54.361 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:04:54.361 INFO:tasks.workunit.client.0.smithi027.stdout: "commit": "f346cc73d15a568d3bec1c99a58953de76f57b85", 2020-09-12T14:04:54.361 INFO:tasks.workunit.client.0.smithi027.stdout: "timestamp": "2020-09-12T14:04:54.358893+0000", 2020-09-12T14:04:54.361 INFO:tasks.workunit.client.0.smithi027.stdout: "tag": "", 2020-09-12T14:04:54.361 INFO:tasks.workunit.client.0.smithi027.stdout: "health": { 2020-09-12T14:04:54.361 INFO:tasks.workunit.client.0.smithi027.stdout: "status": "HEALTH_WARN", 2020-09-12T14:04:54.361 INFO:tasks.workunit.client.0.smithi027.stdout: "checks": { 2020-09-12T14:04:54.362 INFO:tasks.workunit.client.0.smithi027.stdout: "OSDMAP_FLAGS": { 2020-09-12T14:04:54.362 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:04:54.362 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:04:54.362 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "nodown,noout,noin flag(s) set", 2020-09-12T14:04:54.362 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 17 2020-09-12T14:04:54.362 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.362 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [], 2020-09-12T14:04:54.363 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:04:54.363 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.363 INFO:tasks.workunit.client.0.smithi027.stdout: "OSD_DOWN": { 2020-09-12T14:04:54.363 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:04:54.363 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:04:54.363 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "1 osds down", 2020-09-12T14:04:54.363 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 1 2020-09-12T14:04:54.363 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.364 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [ 2020-09-12T14:04:54.364 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.364 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "osd.1 (root=default,host=smithi027) is down" 2020-09-12T14:04:54.364 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.364 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:54.364 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:04:54.364 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.365 INFO:tasks.workunit.client.0.smithi027.stdout: "PG_DEGRADED": { 2020-09-12T14:04:54.365 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:04:54.365 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:04:54.365 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 32 pgs degraded, 32 pgs undersized", 2020-09-12T14:04:54.365 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 64 2020-09-12T14:04:54.365 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.365 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [ 2020-09-12T14:04:54.365 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.366 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.0 is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.366 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.366 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.366 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.1 is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.366 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.366 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.366 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.2 is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.367 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.367 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.367 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.3 is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.367 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.367 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.367 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.4 is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.367 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.367 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.368 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.5 is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.368 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.368 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.368 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.6 is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.368 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.368 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.368 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.7 is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.369 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.369 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.369 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.8 is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.369 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.369 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.369 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.9 is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.369 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.369 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.370 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.a is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.370 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.370 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.370 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.b is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.370 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.370 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.370 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.c is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.371 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.371 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.371 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.d is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.371 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.371 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.371 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.e is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.371 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.371 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.372 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.f is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.372 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.372 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.372 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.10 is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.372 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.372 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.372 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.11 is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.373 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.373 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.373 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.12 is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.373 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.373 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.373 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.13 is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.373 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.373 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.374 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.14 is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.374 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.374 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.374 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.15 is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.374 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.374 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.374 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.16 is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.374 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.375 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.375 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.17 is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.375 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.375 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.375 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.18 is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.375 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.375 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.376 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.19 is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.376 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.376 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.376 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.1a is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.376 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.376 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.376 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.1b is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.376 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.377 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.377 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.1c is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.377 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.377 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.377 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.1d is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.377 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.377 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.377 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.1e is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.378 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.378 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.378 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.1f is stuck undersized for 72s, current state active+undersized+degraded, last acting [0,2]" 2020-09-12T14:04:54.378 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.378 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:54.378 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:04:54.378 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.379 INFO:tasks.workunit.client.0.smithi027.stdout: "POOL_APP_NOT_ENABLED": { 2020-09-12T14:04:54.379 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:04:54.379 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:04:54.379 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "1 pool(s) do not have an application enabled", 2020-09-12T14:04:54.379 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 1 2020-09-12T14:04:54.379 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.379 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [ 2020-09-12T14:04:54.379 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.380 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "application not enabled on pool 'test'" 2020-09-12T14:04:54.380 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.380 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.380 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2020-09-12T14:04:54.380 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.380 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:54.380 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:04:54.380 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.381 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.381 INFO:tasks.workunit.client.0.smithi027.stdout: "mutes": [] 2020-09-12T14:04:54.381 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.381 INFO:tasks.workunit.client.0.smithi027.stdout: "monmap_first_committed": 1, 2020-09-12T14:04:54.381 INFO:tasks.workunit.client.0.smithi027.stdout: "monmap_last_committed": 1, 2020-09-12T14:04:54.381 INFO:tasks.workunit.client.0.smithi027.stdout: "monmap": { 2020-09-12T14:04:54.381 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 1, 2020-09-12T14:04:54.382 INFO:tasks.workunit.client.0.smithi027.stdout: "fsid": "7e90bad3-7973-4a8c-be65-6326ab931fe8", 2020-09-12T14:04:54.382 INFO:tasks.workunit.client.0.smithi027.stdout: "modified": "2020-09-12T14:02:17.274059Z", 2020-09-12T14:04:54.382 INFO:tasks.workunit.client.0.smithi027.stdout: "created": "2020-09-12T14:02:17.274059Z", 2020-09-12T14:04:54.382 INFO:tasks.workunit.client.0.smithi027.stdout: "min_mon_release": 16, 2020-09-12T14:04:54.382 INFO:tasks.workunit.client.0.smithi027.stdout: "min_mon_release_name": "pacific", 2020-09-12T14:04:54.382 INFO:tasks.workunit.client.0.smithi027.stdout: "features": { 2020-09-12T14:04:54.382 INFO:tasks.workunit.client.0.smithi027.stdout: "persistent": [ 2020-09-12T14:04:54.382 INFO:tasks.workunit.client.0.smithi027.stdout: "kraken", 2020-09-12T14:04:54.383 INFO:tasks.workunit.client.0.smithi027.stdout: "luminous", 2020-09-12T14:04:54.383 INFO:tasks.workunit.client.0.smithi027.stdout: "mimic", 2020-09-12T14:04:54.383 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap-prune", 2020-09-12T14:04:54.383 INFO:tasks.workunit.client.0.smithi027.stdout: "nautilus", 2020-09-12T14:04:54.383 INFO:tasks.workunit.client.0.smithi027.stdout: "octopus", 2020-09-12T14:04:54.383 INFO:tasks.workunit.client.0.smithi027.stdout: "pacific" 2020-09-12T14:04:54.383 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:54.384 INFO:tasks.workunit.client.0.smithi027.stdout: "optional": [] 2020-09-12T14:04:54.384 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.384 INFO:tasks.workunit.client.0.smithi027.stdout: "mons": [ 2020-09-12T14:04:54.384 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.384 INFO:tasks.workunit.client.0.smithi027.stdout: "rank": 0, 2020-09-12T14:04:54.384 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "a", 2020-09-12T14:04:54.384 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:04:54.384 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:54.385 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.385 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:54.385 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:7115", 2020-09-12T14:04:54.385 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 0 2020-09-12T14:04:54.385 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.385 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.385 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.385 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:7115/0", 2020-09-12T14:04:54.386 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:7115/0", 2020-09-12T14:04:54.386 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 0, 2020-09-12T14:04:54.386 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 0 2020-09-12T14:04:54.386 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.386 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.386 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.386 INFO:tasks.workunit.client.0.smithi027.stdout: "quorum": [ 2020-09-12T14:04:54.387 INFO:tasks.workunit.client.0.smithi027.stdout: 0 2020-09-12T14:04:54.387 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:54.387 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap": { 2020-09-12T14:04:54.387 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 47, 2020-09-12T14:04:54.387 INFO:tasks.workunit.client.0.smithi027.stdout: "fsid": "7e90bad3-7973-4a8c-be65-6326ab931fe8", 2020-09-12T14:04:54.387 INFO:tasks.workunit.client.0.smithi027.stdout: "created": "2020-09-12T14:02:18.091340+0000", 2020-09-12T14:04:54.387 INFO:tasks.workunit.client.0.smithi027.stdout: "modified": "2020-09-12T14:03:39.453198+0000", 2020-09-12T14:04:54.387 INFO:tasks.workunit.client.0.smithi027.stdout: "last_up_change": "2020-09-12T14:03:37.501958+0000", 2020-09-12T14:04:54.388 INFO:tasks.workunit.client.0.smithi027.stdout: "last_in_change": "2020-09-12T14:03:38.883551+0000", 2020-09-12T14:04:54.388 INFO:tasks.workunit.client.0.smithi027.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-09-12T14:04:54.388 INFO:tasks.workunit.client.0.smithi027.stdout: "flags_num": 5800384, 2020-09-12T14:04:54.388 INFO:tasks.workunit.client.0.smithi027.stdout: "flags_set": [ 2020-09-12T14:04:54.388 INFO:tasks.workunit.client.0.smithi027.stdout: "nodown", 2020-09-12T14:04:54.388 INFO:tasks.workunit.client.0.smithi027.stdout: "noin", 2020-09-12T14:04:54.388 INFO:tasks.workunit.client.0.smithi027.stdout: "noout", 2020-09-12T14:04:54.388 INFO:tasks.workunit.client.0.smithi027.stdout: "pglog_hardlimit", 2020-09-12T14:04:54.389 INFO:tasks.workunit.client.0.smithi027.stdout: "purged_snapdirs", 2020-09-12T14:04:54.389 INFO:tasks.workunit.client.0.smithi027.stdout: "recovery_deletes", 2020-09-12T14:04:54.389 INFO:tasks.workunit.client.0.smithi027.stdout: "sortbitwise" 2020-09-12T14:04:54.389 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:54.389 INFO:tasks.workunit.client.0.smithi027.stdout: "crush_version": 7, 2020-09-12T14:04:54.389 INFO:tasks.workunit.client.0.smithi027.stdout: "full_ratio": 0.99000000953674316, 2020-09-12T14:04:54.389 INFO:tasks.workunit.client.0.smithi027.stdout: "backfillfull_ratio": 0.99000000953674316, 2020-09-12T14:04:54.389 INFO:tasks.workunit.client.0.smithi027.stdout: "nearfull_ratio": 0.85000002384185791, 2020-09-12T14:04:54.390 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_snapshot": "", 2020-09-12T14:04:54.390 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_max": 1, 2020-09-12T14:04:54.390 INFO:tasks.workunit.client.0.smithi027.stdout: "max_osd": 3, 2020-09-12T14:04:54.390 INFO:tasks.workunit.client.0.smithi027.stdout: "require_min_compat_client": "luminous", 2020-09-12T14:04:54.390 INFO:tasks.workunit.client.0.smithi027.stdout: "min_compat_client": "jewel", 2020-09-12T14:04:54.390 INFO:tasks.workunit.client.0.smithi027.stdout: "require_osd_release": "pacific", 2020-09-12T14:04:54.390 INFO:tasks.workunit.client.0.smithi027.stdout: "pools": [ 2020-09-12T14:04:54.391 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.391 INFO:tasks.workunit.client.0.smithi027.stdout: "pool": 1, 2020-09-12T14:04:54.391 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_name": "test", 2020-09-12T14:04:54.391 INFO:tasks.workunit.client.0.smithi027.stdout: "create_time": "2020-09-12T14:02:45.510238+0000", 2020-09-12T14:04:54.391 INFO:tasks.workunit.client.0.smithi027.stdout: "flags": 1, 2020-09-12T14:04:54.391 INFO:tasks.workunit.client.0.smithi027.stdout: "flags_names": "hashpspool", 2020-09-12T14:04:54.391 INFO:tasks.workunit.client.0.smithi027.stdout: "type": 1, 2020-09-12T14:04:54.391 INFO:tasks.workunit.client.0.smithi027.stdout: "size": 3, 2020-09-12T14:04:54.392 INFO:tasks.workunit.client.0.smithi027.stdout: "min_size": 2, 2020-09-12T14:04:54.392 INFO:tasks.workunit.client.0.smithi027.stdout: "crush_rule": 0, 2020-09-12T14:04:54.392 INFO:tasks.workunit.client.0.smithi027.stdout: "object_hash": 2, 2020-09-12T14:04:54.392 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_autoscale_mode": "on", 2020-09-12T14:04:54.392 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_num": 32, 2020-09-12T14:04:54.392 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_placement_num": 1, 2020-09-12T14:04:54.392 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_placement_num_target": 32, 2020-09-12T14:04:54.392 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_num_target": 32, 2020-09-12T14:04:54.393 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_num_pending": 32, 2020-09-12T14:04:54.393 INFO:tasks.workunit.client.0.smithi027.stdout: "last_pg_merge_meta": { 2020-09-12T14:04:54.393 INFO:tasks.workunit.client.0.smithi027.stdout: "source_pgid": "0.0", 2020-09-12T14:04:54.393 INFO:tasks.workunit.client.0.smithi027.stdout: "ready_epoch": 0, 2020-09-12T14:04:54.393 INFO:tasks.workunit.client.0.smithi027.stdout: "last_epoch_started": 0, 2020-09-12T14:04:54.393 INFO:tasks.workunit.client.0.smithi027.stdout: "last_epoch_clean": 0, 2020-09-12T14:04:54.393 INFO:tasks.workunit.client.0.smithi027.stdout: "source_version": "0'0", 2020-09-12T14:04:54.393 INFO:tasks.workunit.client.0.smithi027.stdout: "target_version": "0'0" 2020-09-12T14:04:54.394 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.394 INFO:tasks.workunit.client.0.smithi027.stdout: "last_change": "33", 2020-09-12T14:04:54.394 INFO:tasks.workunit.client.0.smithi027.stdout: "last_force_op_resend": "0", 2020-09-12T14:04:54.394 INFO:tasks.workunit.client.0.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2020-09-12T14:04:54.394 INFO:tasks.workunit.client.0.smithi027.stdout: "last_force_op_resend_preluminous": "33", 2020-09-12T14:04:54.394 INFO:tasks.workunit.client.0.smithi027.stdout: "auid": 0, 2020-09-12T14:04:54.394 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_mode": "selfmanaged", 2020-09-12T14:04:54.395 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_seq": 0, 2020-09-12T14:04:54.395 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_epoch": 0, 2020-09-12T14:04:54.395 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_snaps": [], 2020-09-12T14:04:54.395 INFO:tasks.workunit.client.0.smithi027.stdout: "removed_snaps": "[]", 2020-09-12T14:04:54.395 INFO:tasks.workunit.client.0.smithi027.stdout: "quota_max_bytes": 0, 2020-09-12T14:04:54.395 INFO:tasks.workunit.client.0.smithi027.stdout: "quota_max_objects": 0, 2020-09-12T14:04:54.395 INFO:tasks.workunit.client.0.smithi027.stdout: "tiers": [], 2020-09-12T14:04:54.395 INFO:tasks.workunit.client.0.smithi027.stdout: "tier_of": -1, 2020-09-12T14:04:54.396 INFO:tasks.workunit.client.0.smithi027.stdout: "read_tier": -1, 2020-09-12T14:04:54.396 INFO:tasks.workunit.client.0.smithi027.stdout: "write_tier": -1, 2020-09-12T14:04:54.396 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_mode": "none", 2020-09-12T14:04:54.396 INFO:tasks.workunit.client.0.smithi027.stdout: "target_max_bytes": 0, 2020-09-12T14:04:54.396 INFO:tasks.workunit.client.0.smithi027.stdout: "target_max_objects": 0, 2020-09-12T14:04:54.396 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-09-12T14:04:54.396 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-09-12T14:04:54.397 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2020-09-12T14:04:54.397 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_min_flush_age": 0, 2020-09-12T14:04:54.397 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_min_evict_age": 0, 2020-09-12T14:04:54.397 INFO:tasks.workunit.client.0.smithi027.stdout: "erasure_code_profile": "", 2020-09-12T14:04:54.397 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_params": { 2020-09-12T14:04:54.397 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "none" 2020-09-12T14:04:54.397 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.397 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_period": 0, 2020-09-12T14:04:54.398 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_count": 0, 2020-09-12T14:04:54.398 INFO:tasks.workunit.client.0.smithi027.stdout: "use_gmt_hitset": true, 2020-09-12T14:04:54.398 INFO:tasks.workunit.client.0.smithi027.stdout: "min_read_recency_for_promote": 0, 2020-09-12T14:04:54.398 INFO:tasks.workunit.client.0.smithi027.stdout: "min_write_recency_for_promote": 0, 2020-09-12T14:04:54.398 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2020-09-12T14:04:54.398 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_search_last_n": 0, 2020-09-12T14:04:54.398 INFO:tasks.workunit.client.0.smithi027.stdout: "grade_table": [], 2020-09-12T14:04:54.398 INFO:tasks.workunit.client.0.smithi027.stdout: "stripe_width": 0, 2020-09-12T14:04:54.399 INFO:tasks.workunit.client.0.smithi027.stdout: "expected_num_objects": 0, 2020-09-12T14:04:54.399 INFO:tasks.workunit.client.0.smithi027.stdout: "fast_read": false, 2020-09-12T14:04:54.399 INFO:tasks.workunit.client.0.smithi027.stdout: "options": {}, 2020-09-12T14:04:54.399 INFO:tasks.workunit.client.0.smithi027.stdout: "application_metadata": {} 2020-09-12T14:04:54.399 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.399 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:54.399 INFO:tasks.workunit.client.0.smithi027.stdout: "osds": [ 2020-09-12T14:04:54.400 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.400 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:04:54.400 INFO:tasks.workunit.client.0.smithi027.stdout: "uuid": "9d8d2ee9-0710-4a4e-a0ba-cec45cbf29a2", 2020-09-12T14:04:54.400 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 1, 2020-09-12T14:04:54.400 INFO:tasks.workunit.client.0.smithi027.stdout: "in": 1, 2020-09-12T14:04:54.400 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 1, 2020-09-12T14:04:54.400 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_affinity": 1, 2020-09-12T14:04:54.401 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_begin": 5, 2020-09-12T14:04:54.401 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_end": 31, 2020-09-12T14:04:54.401 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 40, 2020-09-12T14:04:54.401 INFO:tasks.workunit.client.0.smithi027.stdout: "up_thru": 45, 2020-09-12T14:04:54.401 INFO:tasks.workunit.client.0.smithi027.stdout: "down_at": 35, 2020-09-12T14:04:54.401 INFO:tasks.workunit.client.0.smithi027.stdout: "lost_at": 0, 2020-09-12T14:04:54.401 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:04:54.401 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:54.402 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.402 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:54.402 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6802", 2020-09-12T14:04:54.402 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49081 2020-09-12T14:04:54.402 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.402 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.402 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:54.403 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6803", 2020-09-12T14:04:54.403 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49081 2020-09-12T14:04:54.403 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.403 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.403 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.403 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addrs": { 2020-09-12T14:04:54.403 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:54.403 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.404 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:54.404 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6804", 2020-09-12T14:04:54.404 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49081 2020-09-12T14:04:54.404 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.404 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.404 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:54.404 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6805", 2020-09-12T14:04:54.404 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49081 2020-09-12T14:04:54.405 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.405 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.405 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.405 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addrs": { 2020-09-12T14:04:54.405 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:54.405 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.405 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:54.405 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6808", 2020-09-12T14:04:54.406 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49081 2020-09-12T14:04:54.406 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.406 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.406 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:54.406 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6809", 2020-09-12T14:04:54.406 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49081 2020-09-12T14:04:54.406 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.407 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.407 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.407 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addrs": { 2020-09-12T14:04:54.407 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:54.407 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.407 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:54.407 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6806", 2020-09-12T14:04:54.407 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49081 2020-09-12T14:04:54.408 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.408 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.408 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:54.408 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6807", 2020-09-12T14:04:54.408 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49081 2020-09-12T14:04:54.408 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.408 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.408 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.409 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:6803/49081", 2020-09-12T14:04:54.409 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addr": "127.0.0.1:6805/49081", 2020-09-12T14:04:54.409 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addr": "127.0.0.1:6809/49081", 2020-09-12T14:04:54.409 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addr": "127.0.0.1:6807/49081", 2020-09-12T14:04:54.409 INFO:tasks.workunit.client.0.smithi027.stdout: "state": [ 2020-09-12T14:04:54.409 INFO:tasks.workunit.client.0.smithi027.stdout: "exists", 2020-09-12T14:04:54.409 INFO:tasks.workunit.client.0.smithi027.stdout: "up" 2020-09-12T14:04:54.410 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.410 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.410 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.410 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:04:54.410 INFO:tasks.workunit.client.0.smithi027.stdout: "uuid": "06975646-fe2b-4ca5-82e0-a2d0e01e06a8", 2020-09-12T14:04:54.410 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 0, 2020-09-12T14:04:54.410 INFO:tasks.workunit.client.0.smithi027.stdout: "in": 1, 2020-09-12T14:04:54.410 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 1, 2020-09-12T14:04:54.411 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_affinity": 1, 2020-09-12T14:04:54.411 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_begin": 0, 2020-09-12T14:04:54.411 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_end": 0, 2020-09-12T14:04:54.411 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 10, 2020-09-12T14:04:54.411 INFO:tasks.workunit.client.0.smithi027.stdout: "up_thru": 33, 2020-09-12T14:04:54.411 INFO:tasks.workunit.client.0.smithi027.stdout: "down_at": 36, 2020-09-12T14:04:54.411 INFO:tasks.workunit.client.0.smithi027.stdout: "lost_at": 0, 2020-09-12T14:04:54.411 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:04:54.412 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:54.412 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.412 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:54.412 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6810", 2020-09-12T14:04:54.412 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 44126 2020-09-12T14:04:54.412 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.412 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.412 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:54.413 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6811", 2020-09-12T14:04:54.413 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 44126 2020-09-12T14:04:54.413 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.413 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.413 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.413 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addrs": { 2020-09-12T14:04:54.413 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:54.414 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.414 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:54.414 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6812", 2020-09-12T14:04:54.414 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 44126 2020-09-12T14:04:54.414 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.414 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.414 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:54.414 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6813", 2020-09-12T14:04:54.415 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 44126 2020-09-12T14:04:54.415 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.415 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.415 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.415 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addrs": { 2020-09-12T14:04:54.415 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:54.415 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.415 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:54.416 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6816", 2020-09-12T14:04:54.416 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 44126 2020-09-12T14:04:54.416 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.416 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.416 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:54.416 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6817", 2020-09-12T14:04:54.416 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 44126 2020-09-12T14:04:54.417 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.417 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.417 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.417 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addrs": { 2020-09-12T14:04:54.417 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:54.417 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.417 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:54.417 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6814", 2020-09-12T14:04:54.418 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 44126 2020-09-12T14:04:54.418 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.418 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.418 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:54.418 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6815", 2020-09-12T14:04:54.418 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 44126 2020-09-12T14:04:54.418 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.418 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.419 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.419 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:6811/44126", 2020-09-12T14:04:54.419 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addr": "127.0.0.1:6813/44126", 2020-09-12T14:04:54.419 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addr": "127.0.0.1:6817/44126", 2020-09-12T14:04:54.419 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addr": "127.0.0.1:6815/44126", 2020-09-12T14:04:54.419 INFO:tasks.workunit.client.0.smithi027.stdout: "state": [ 2020-09-12T14:04:54.419 INFO:tasks.workunit.client.0.smithi027.stdout: "exists" 2020-09-12T14:04:54.419 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.420 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.420 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.420 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:04:54.420 INFO:tasks.workunit.client.0.smithi027.stdout: "uuid": "ed2b43d9-2a50-45c4-b6a7-8f00f86304e8", 2020-09-12T14:04:54.420 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 1, 2020-09-12T14:04:54.420 INFO:tasks.workunit.client.0.smithi027.stdout: "in": 1, 2020-09-12T14:04:54.420 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 1, 2020-09-12T14:04:54.421 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_affinity": 1, 2020-09-12T14:04:54.421 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_begin": 15, 2020-09-12T14:04:54.421 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_end": 31, 2020-09-12T14:04:54.421 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 41, 2020-09-12T14:04:54.421 INFO:tasks.workunit.client.0.smithi027.stdout: "up_thru": 0, 2020-09-12T14:04:54.421 INFO:tasks.workunit.client.0.smithi027.stdout: "down_at": 37, 2020-09-12T14:04:54.421 INFO:tasks.workunit.client.0.smithi027.stdout: "lost_at": 0, 2020-09-12T14:04:54.421 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:04:54.422 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:54.422 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.422 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:54.422 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6810", 2020-09-12T14:04:54.422 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49480 2020-09-12T14:04:54.422 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.422 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.422 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:54.423 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6811", 2020-09-12T14:04:54.423 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49480 2020-09-12T14:04:54.423 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.423 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.423 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.423 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addrs": { 2020-09-12T14:04:54.423 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:54.424 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.424 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:54.424 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6812", 2020-09-12T14:04:54.424 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49480 2020-09-12T14:04:54.424 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.424 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.424 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:54.424 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6813", 2020-09-12T14:04:54.425 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49480 2020-09-12T14:04:54.425 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.425 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.425 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.425 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addrs": { 2020-09-12T14:04:54.425 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:54.425 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.425 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:54.426 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6816", 2020-09-12T14:04:54.426 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49480 2020-09-12T14:04:54.426 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.426 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.426 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:54.426 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6817", 2020-09-12T14:04:54.426 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49480 2020-09-12T14:04:54.426 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.427 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.427 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.427 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addrs": { 2020-09-12T14:04:54.427 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:04:54.427 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.427 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:04:54.427 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6814", 2020-09-12T14:04:54.428 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49480 2020-09-12T14:04:54.428 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.428 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.428 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:04:54.428 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6815", 2020-09-12T14:04:54.428 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 49480 2020-09-12T14:04:54.428 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.428 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.429 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.429 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:6811/49480", 2020-09-12T14:04:54.429 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addr": "127.0.0.1:6813/49480", 2020-09-12T14:04:54.429 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addr": "127.0.0.1:6817/49480", 2020-09-12T14:04:54.429 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addr": "127.0.0.1:6815/49480", 2020-09-12T14:04:54.429 INFO:tasks.workunit.client.0.smithi027.stdout: "state": [ 2020-09-12T14:04:54.429 INFO:tasks.workunit.client.0.smithi027.stdout: "exists", 2020-09-12T14:04:54.429 INFO:tasks.workunit.client.0.smithi027.stdout: "up" 2020-09-12T14:04:54.430 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.430 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.430 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:54.430 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_xinfo": [ 2020-09-12T14:04:54.430 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.430 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:04:54.430 INFO:tasks.workunit.client.0.smithi027.stdout: "down_stamp": "2020-09-12T14:03:30.043982+0000", 2020-09-12T14:04:54.430 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_probability": 0, 2020-09-12T14:04:54.431 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_interval": 0, 2020-09-12T14:04:54.431 INFO:tasks.workunit.client.0.smithi027.stdout: "features": 4540138292841938943, 2020-09-12T14:04:54.431 INFO:tasks.workunit.client.0.smithi027.stdout: "old_weight": 0, 2020-09-12T14:04:54.431 INFO:tasks.workunit.client.0.smithi027.stdout: "last_purged_snaps_scrub": "2020-09-12T14:02:26.580267+0000", 2020-09-12T14:04:54.431 INFO:tasks.workunit.client.0.smithi027.stdout: "dead_epoch": 0 2020-09-12T14:04:54.431 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.431 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.432 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:04:54.432 INFO:tasks.workunit.client.0.smithi027.stdout: "down_stamp": "2020-09-12T14:03:30.629689+0000", 2020-09-12T14:04:54.432 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_probability": 0, 2020-09-12T14:04:54.432 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_interval": 0, 2020-09-12T14:04:54.432 INFO:tasks.workunit.client.0.smithi027.stdout: "features": 4540138292841938943, 2020-09-12T14:04:54.432 INFO:tasks.workunit.client.0.smithi027.stdout: "old_weight": 0, 2020-09-12T14:04:54.432 INFO:tasks.workunit.client.0.smithi027.stdout: "last_purged_snaps_scrub": "2020-09-12T14:02:34.628567+0000", 2020-09-12T14:04:54.432 INFO:tasks.workunit.client.0.smithi027.stdout: "dead_epoch": 0 2020-09-12T14:04:54.433 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.433 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.433 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:04:54.433 INFO:tasks.workunit.client.0.smithi027.stdout: "down_stamp": "2020-09-12T14:03:31.229169+0000", 2020-09-12T14:04:54.433 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_probability": 0, 2020-09-12T14:04:54.433 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_interval": 0, 2020-09-12T14:04:54.433 INFO:tasks.workunit.client.0.smithi027.stdout: "features": 4540138292841938943, 2020-09-12T14:04:54.433 INFO:tasks.workunit.client.0.smithi027.stdout: "old_weight": 0, 2020-09-12T14:04:54.434 INFO:tasks.workunit.client.0.smithi027.stdout: "last_purged_snaps_scrub": "2020-09-12T14:02:42.728129+0000", 2020-09-12T14:04:54.434 INFO:tasks.workunit.client.0.smithi027.stdout: "dead_epoch": 0 2020-09-12T14:04:54.434 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.434 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:54.434 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_upmap": [], 2020-09-12T14:04:54.434 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_upmap_items": [], 2020-09-12T14:04:54.434 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_temp": [], 2020-09-12T14:04:54.434 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_temp": [], 2020-09-12T14:04:54.435 INFO:tasks.workunit.client.0.smithi027.stdout: "blocklist": {}, 2020-09-12T14:04:54.435 INFO:tasks.workunit.client.0.smithi027.stdout: "erasure_code_profiles": { 2020-09-12T14:04:54.435 INFO:tasks.workunit.client.0.smithi027.stdout: "default": { 2020-09-12T14:04:54.435 INFO:tasks.workunit.client.0.smithi027.stdout: "k": "2", 2020-09-12T14:04:54.435 INFO:tasks.workunit.client.0.smithi027.stdout: "m": "2", 2020-09-12T14:04:54.435 INFO:tasks.workunit.client.0.smithi027.stdout: "plugin": "jerasure", 2020-09-12T14:04:54.435 INFO:tasks.workunit.client.0.smithi027.stdout: "technique": "reed_sol_van" 2020-09-12T14:04:54.435 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.436 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.436 INFO:tasks.workunit.client.0.smithi027.stdout: "removed_snaps_queue": [], 2020-09-12T14:04:54.436 INFO:tasks.workunit.client.0.smithi027.stdout: "new_removed_snaps": [], 2020-09-12T14:04:54.436 INFO:tasks.workunit.client.0.smithi027.stdout: "new_purged_snaps": [], 2020-09-12T14:04:54.436 INFO:tasks.workunit.client.0.smithi027.stdout: "crush_node_flags": {}, 2020-09-12T14:04:54.436 INFO:tasks.workunit.client.0.smithi027.stdout: "device_class_flags": {} 2020-09-12T14:04:54.436 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.437 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_metadata": [ 2020-09-12T14:04:54.437 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.437 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:04:54.437 INFO:tasks.workunit.client.0.smithi027.stdout: "arch": "x86_64", 2020-09-12T14:04:54.437 INFO:tasks.workunit.client.0.smithi027.stdout: "back_addr": "[v2:127.0.0.1:6804/49081,v1:127.0.0.1:6805/49081]", 2020-09-12T14:04:54.437 INFO:tasks.workunit.client.0.smithi027.stdout: "back_iface": "", 2020-09-12T14:04:54.437 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs": "1", 2020-09-12T14:04:54.437 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_db": "0", 2020-09-12T14:04:54.438 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_wal": "0", 2020-09-12T14:04:54.438 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_single_shared_device": "1", 2020-09-12T14:04:54.438 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_access_mode": "file", 2020-09-12T14:04:54.438 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_block_size": "4096", 2020-09-12T14:04:54.438 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_devices": "sda", 2020-09-12T14:04:54.438 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_driver": "KernelDevice", 2020-09-12T14:04:54.438 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2020-09-12T14:04:54.438 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_rotational": "1", 2020-09-12T14:04:54.439 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_size": "107374182400", 2020-09-12T14:04:54.439 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_support_discard": "0", 2020-09-12T14:04:54.439 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_type": "hdd", 2020-09-12T14:04:54.439 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_release": "pacific", 2020-09-12T14:04:54.439 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version": "ceph version 16.0.0-5485-gf346cc73d15 (f346cc73d15a568d3bec1c99a58953de76f57b85) pacific (dev)", 2020-09-12T14:04:54.439 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version_short": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:04:54.439 INFO:tasks.workunit.client.0.smithi027.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2020-09-12T14:04:54.439 INFO:tasks.workunit.client.0.smithi027.stdout: "default_device_class": "hdd", 2020-09-12T14:04:54.440 INFO:tasks.workunit.client.0.smithi027.stdout: "device_ids": "sda=ST1000NM0033-9ZM173_Z1W4H3N7", 2020-09-12T14:04:54.440 INFO:tasks.workunit.client.0.smithi027.stdout: "device_paths": "sda=/dev/disk/by-path/pci-0000:00:1f.2-ata-1", 2020-09-12T14:04:54.440 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": "sda", 2020-09-12T14:04:54.440 INFO:tasks.workunit.client.0.smithi027.stdout: "distro": "centos", 2020-09-12T14:04:54.440 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_description": "CentOS Linux 8 (Core)", 2020-09-12T14:04:54.440 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_version": "8", 2020-09-12T14:04:54.440 INFO:tasks.workunit.client.0.smithi027.stdout: "front_addr": "[v2:127.0.0.1:6802/49081,v1:127.0.0.1:6803/49081]", 2020-09-12T14:04:54.441 INFO:tasks.workunit.client.0.smithi027.stdout: "front_iface": "", 2020-09-12T14:04:54.441 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/49081,v1:127.0.0.1:6809/49081]", 2020-09-12T14:04:54.441 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/49081,v1:127.0.0.1:6807/49081]", 2020-09-12T14:04:54.441 INFO:tasks.workunit.client.0.smithi027.stdout: "hostname": "smithi027", 2020-09-12T14:04:54.441 INFO:tasks.workunit.client.0.smithi027.stdout: "journal_rotational": "1", 2020-09-12T14:04:54.441 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_description": "#1 SMP Sun Jul 26 03:54:29 UTC 2020", 2020-09-12T14:04:54.441 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_version": "4.18.0-193.14.2.el8_2.x86_64", 2020-09-12T14:04:54.441 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_swap_kb": "0", 2020-09-12T14:04:54.442 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_total_kb": "32659336", 2020-09-12T14:04:54.442 INFO:tasks.workunit.client.0.smithi027.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2020-09-12T14:04:54.442 INFO:tasks.workunit.client.0.smithi027.stdout: "objectstore_numa_unknown_devices": "sda", 2020-09-12T14:04:54.442 INFO:tasks.workunit.client.0.smithi027.stdout: "os": "Linux", 2020-09-12T14:04:54.442 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_data": "td/divergent-priors/0", 2020-09-12T14:04:54.442 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_objectstore": "bluestore", 2020-09-12T14:04:54.442 INFO:tasks.workunit.client.0.smithi027.stdout: "osdspec_affinity": "", 2020-09-12T14:04:54.442 INFO:tasks.workunit.client.0.smithi027.stdout: "rotational": "1" 2020-09-12T14:04:54.443 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.443 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.443 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:04:54.443 INFO:tasks.workunit.client.0.smithi027.stdout: "arch": "x86_64", 2020-09-12T14:04:54.443 INFO:tasks.workunit.client.0.smithi027.stdout: "back_addr": "[v2:127.0.0.1:6812/44126,v1:127.0.0.1:6813/44126]", 2020-09-12T14:04:54.443 INFO:tasks.workunit.client.0.smithi027.stdout: "back_iface": "", 2020-09-12T14:04:54.443 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs": "1", 2020-09-12T14:04:54.444 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_db": "0", 2020-09-12T14:04:54.444 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_wal": "0", 2020-09-12T14:04:54.444 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_single_shared_device": "1", 2020-09-12T14:04:54.444 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_access_mode": "file", 2020-09-12T14:04:54.444 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_block_size": "4096", 2020-09-12T14:04:54.444 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_devices": "sda", 2020-09-12T14:04:54.444 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_driver": "KernelDevice", 2020-09-12T14:04:54.444 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2020-09-12T14:04:54.445 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_rotational": "1", 2020-09-12T14:04:54.445 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_size": "107374182400", 2020-09-12T14:04:54.445 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_support_discard": "0", 2020-09-12T14:04:54.445 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_type": "hdd", 2020-09-12T14:04:54.445 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_release": "pacific", 2020-09-12T14:04:54.445 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version": "ceph version 16.0.0-5485-gf346cc73d15 (f346cc73d15a568d3bec1c99a58953de76f57b85) pacific (dev)", 2020-09-12T14:04:54.445 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version_short": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:04:54.445 INFO:tasks.workunit.client.0.smithi027.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2020-09-12T14:04:54.446 INFO:tasks.workunit.client.0.smithi027.stdout: "default_device_class": "hdd", 2020-09-12T14:04:54.446 INFO:tasks.workunit.client.0.smithi027.stdout: "device_ids": "sda=ST1000NM0033-9ZM173_Z1W4H3N7", 2020-09-12T14:04:54.446 INFO:tasks.workunit.client.0.smithi027.stdout: "device_paths": "sda=/dev/disk/by-path/pci-0000:00:1f.2-ata-1", 2020-09-12T14:04:54.446 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": "sda", 2020-09-12T14:04:54.446 INFO:tasks.workunit.client.0.smithi027.stdout: "distro": "centos", 2020-09-12T14:04:54.446 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_description": "CentOS Linux 8 (Core)", 2020-09-12T14:04:54.446 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_version": "8", 2020-09-12T14:04:54.446 INFO:tasks.workunit.client.0.smithi027.stdout: "front_addr": "[v2:127.0.0.1:6810/44126,v1:127.0.0.1:6811/44126]", 2020-09-12T14:04:54.447 INFO:tasks.workunit.client.0.smithi027.stdout: "front_iface": "", 2020-09-12T14:04:54.447 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/44126,v1:127.0.0.1:6817/44126]", 2020-09-12T14:04:54.447 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/44126,v1:127.0.0.1:6815/44126]", 2020-09-12T14:04:54.447 INFO:tasks.workunit.client.0.smithi027.stdout: "hostname": "smithi027", 2020-09-12T14:04:54.447 INFO:tasks.workunit.client.0.smithi027.stdout: "journal_rotational": "1", 2020-09-12T14:04:54.447 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_description": "#1 SMP Sun Jul 26 03:54:29 UTC 2020", 2020-09-12T14:04:54.447 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_version": "4.18.0-193.14.2.el8_2.x86_64", 2020-09-12T14:04:54.448 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_swap_kb": "0", 2020-09-12T14:04:54.448 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_total_kb": "32659336", 2020-09-12T14:04:54.448 INFO:tasks.workunit.client.0.smithi027.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2020-09-12T14:04:54.448 INFO:tasks.workunit.client.0.smithi027.stdout: "objectstore_numa_unknown_devices": "sda", 2020-09-12T14:04:54.448 INFO:tasks.workunit.client.0.smithi027.stdout: "os": "Linux", 2020-09-12T14:04:54.448 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_data": "td/divergent-priors/1", 2020-09-12T14:04:54.448 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_objectstore": "bluestore", 2020-09-12T14:04:54.448 INFO:tasks.workunit.client.0.smithi027.stdout: "osdspec_affinity": "", 2020-09-12T14:04:54.449 INFO:tasks.workunit.client.0.smithi027.stdout: "rotational": "1" 2020-09-12T14:04:54.449 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.449 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.449 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:04:54.449 INFO:tasks.workunit.client.0.smithi027.stdout: "arch": "x86_64", 2020-09-12T14:04:54.449 INFO:tasks.workunit.client.0.smithi027.stdout: "back_addr": "[v2:127.0.0.1:6812/49480,v1:127.0.0.1:6813/49480]", 2020-09-12T14:04:54.449 INFO:tasks.workunit.client.0.smithi027.stdout: "back_iface": "", 2020-09-12T14:04:54.449 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs": "1", 2020-09-12T14:04:54.450 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_db": "0", 2020-09-12T14:04:54.450 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_wal": "0", 2020-09-12T14:04:54.450 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_single_shared_device": "1", 2020-09-12T14:04:54.450 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_access_mode": "file", 2020-09-12T14:04:54.450 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_block_size": "4096", 2020-09-12T14:04:54.450 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_devices": "sda", 2020-09-12T14:04:54.450 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_driver": "KernelDevice", 2020-09-12T14:04:54.450 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2020-09-12T14:04:54.451 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_rotational": "1", 2020-09-12T14:04:54.451 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_size": "107374182400", 2020-09-12T14:04:54.451 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_support_discard": "0", 2020-09-12T14:04:54.451 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_type": "hdd", 2020-09-12T14:04:54.451 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_release": "pacific", 2020-09-12T14:04:54.451 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version": "ceph version 16.0.0-5485-gf346cc73d15 (f346cc73d15a568d3bec1c99a58953de76f57b85) pacific (dev)", 2020-09-12T14:04:54.451 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version_short": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:04:54.452 INFO:tasks.workunit.client.0.smithi027.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2020-09-12T14:04:54.452 INFO:tasks.workunit.client.0.smithi027.stdout: "default_device_class": "hdd", 2020-09-12T14:04:54.452 INFO:tasks.workunit.client.0.smithi027.stdout: "device_ids": "sda=ST1000NM0033-9ZM173_Z1W4H3N7", 2020-09-12T14:04:54.452 INFO:tasks.workunit.client.0.smithi027.stdout: "device_paths": "sda=/dev/disk/by-path/pci-0000:00:1f.2-ata-1", 2020-09-12T14:04:54.452 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": "sda", 2020-09-12T14:04:54.452 INFO:tasks.workunit.client.0.smithi027.stdout: "distro": "centos", 2020-09-12T14:04:54.452 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_description": "CentOS Linux 8 (Core)", 2020-09-12T14:04:54.452 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_version": "8", 2020-09-12T14:04:54.453 INFO:tasks.workunit.client.0.smithi027.stdout: "front_addr": "[v2:127.0.0.1:6810/49480,v1:127.0.0.1:6811/49480]", 2020-09-12T14:04:54.453 INFO:tasks.workunit.client.0.smithi027.stdout: "front_iface": "", 2020-09-12T14:04:54.453 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/49480,v1:127.0.0.1:6817/49480]", 2020-09-12T14:04:54.453 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/49480,v1:127.0.0.1:6815/49480]", 2020-09-12T14:04:54.453 INFO:tasks.workunit.client.0.smithi027.stdout: "hostname": "smithi027", 2020-09-12T14:04:54.453 INFO:tasks.workunit.client.0.smithi027.stdout: "journal_rotational": "1", 2020-09-12T14:04:54.453 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_description": "#1 SMP Sun Jul 26 03:54:29 UTC 2020", 2020-09-12T14:04:54.453 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_version": "4.18.0-193.14.2.el8_2.x86_64", 2020-09-12T14:04:54.454 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_swap_kb": "0", 2020-09-12T14:04:54.454 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_total_kb": "32659336", 2020-09-12T14:04:54.454 INFO:tasks.workunit.client.0.smithi027.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2020-09-12T14:04:54.454 INFO:tasks.workunit.client.0.smithi027.stdout: "objectstore_numa_unknown_devices": "sda", 2020-09-12T14:04:54.454 INFO:tasks.workunit.client.0.smithi027.stdout: "os": "Linux", 2020-09-12T14:04:54.454 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_data": "td/divergent-priors/2", 2020-09-12T14:04:54.454 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_objectstore": "bluestore", 2020-09-12T14:04:54.454 INFO:tasks.workunit.client.0.smithi027.stdout: "osdspec_affinity": "", 2020-09-12T14:04:54.455 INFO:tasks.workunit.client.0.smithi027.stdout: "rotational": "1" 2020-09-12T14:04:54.455 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.455 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:54.455 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap_clean_epochs": { 2020-09-12T14:04:54.455 INFO:tasks.workunit.client.0.smithi027.stdout: "min_last_epoch_clean": 0, 2020-09-12T14:04:54.455 INFO:tasks.workunit.client.0.smithi027.stdout: "last_epoch_clean": { 2020-09-12T14:04:54.455 INFO:tasks.workunit.client.0.smithi027.stdout: "per_pool": [] 2020-09-12T14:04:54.456 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.456 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_epochs": [ 2020-09-12T14:04:54.456 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.456 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:04:54.456 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 40 2020-09-12T14:04:54.456 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.456 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.456 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:04:54.457 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 10 2020-09-12T14:04:54.457 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.457 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.457 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:04:54.457 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 41 2020-09-12T14:04:54.457 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.457 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.457 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.458 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap_first_committed": 1, 2020-09-12T14:04:54.458 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap_last_committed": 47, 2020-09-12T14:04:54.458 INFO:tasks.workunit.client.0.smithi027.stdout: "crushmap": { 2020-09-12T14:04:54.458 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": [ 2020-09-12T14:04:54.458 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.458 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:04:54.458 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd.0", 2020-09-12T14:04:54.458 INFO:tasks.workunit.client.0.smithi027.stdout: "class": "hdd" 2020-09-12T14:04:54.459 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.459 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.459 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:04:54.459 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd.1", 2020-09-12T14:04:54.459 INFO:tasks.workunit.client.0.smithi027.stdout: "class": "hdd" 2020-09-12T14:04:54.459 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.459 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.460 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:04:54.460 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd.2", 2020-09-12T14:04:54.460 INFO:tasks.workunit.client.0.smithi027.stdout: "class": "hdd" 2020-09-12T14:04:54.460 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.460 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:54.460 INFO:tasks.workunit.client.0.smithi027.stdout: "types": [ 2020-09-12T14:04:54.460 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.460 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 0, 2020-09-12T14:04:54.461 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd" 2020-09-12T14:04:54.461 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.461 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.461 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 1, 2020-09-12T14:04:54.461 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "host" 2020-09-12T14:04:54.461 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.461 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.461 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 2, 2020-09-12T14:04:54.462 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "chassis" 2020-09-12T14:04:54.462 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.462 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.462 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 3, 2020-09-12T14:04:54.462 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "rack" 2020-09-12T14:04:54.462 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.462 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.462 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 4, 2020-09-12T14:04:54.463 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "row" 2020-09-12T14:04:54.463 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.463 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.463 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 5, 2020-09-12T14:04:54.463 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "pdu" 2020-09-12T14:04:54.463 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.463 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.463 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 6, 2020-09-12T14:04:54.464 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "pod" 2020-09-12T14:04:54.464 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.464 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.464 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 7, 2020-09-12T14:04:54.464 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "room" 2020-09-12T14:04:54.464 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.464 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.464 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 8, 2020-09-12T14:04:54.465 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "datacenter" 2020-09-12T14:04:54.465 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.465 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.465 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 9, 2020-09-12T14:04:54.465 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "zone" 2020-09-12T14:04:54.465 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.465 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.466 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 10, 2020-09-12T14:04:54.466 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "region" 2020-09-12T14:04:54.466 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.466 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.466 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 11, 2020-09-12T14:04:54.466 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "root" 2020-09-12T14:04:54.466 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.466 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:54.467 INFO:tasks.workunit.client.0.smithi027.stdout: "buckets": [ 2020-09-12T14:04:54.467 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.467 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -1, 2020-09-12T14:04:54.467 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "default", 2020-09-12T14:04:54.467 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 11, 2020-09-12T14:04:54.467 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "root", 2020-09-12T14:04:54.467 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:04:54.467 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:04:54.468 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:04:54.468 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:04:54.468 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.468 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -3, 2020-09-12T14:04:54.468 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:04:54.468 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:04:54.468 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.468 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.469 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.469 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.469 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -2, 2020-09-12T14:04:54.469 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "default~hdd", 2020-09-12T14:04:54.469 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 11, 2020-09-12T14:04:54.469 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "root", 2020-09-12T14:04:54.469 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:04:54.469 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:04:54.470 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:04:54.470 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:04:54.470 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.470 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -4, 2020-09-12T14:04:54.470 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:04:54.470 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:04:54.470 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.471 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.471 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.471 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.471 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -3, 2020-09-12T14:04:54.471 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "smithi027", 2020-09-12T14:04:54.471 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 1, 2020-09-12T14:04:54.471 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "host", 2020-09-12T14:04:54.471 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:04:54.472 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:04:54.472 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:04:54.472 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:04:54.472 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.472 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:04:54.472 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:04:54.472 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:04:54.473 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.473 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.473 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:04:54.473 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:04:54.473 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 1 2020-09-12T14:04:54.473 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.473 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.473 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:04:54.474 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:04:54.474 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 2 2020-09-12T14:04:54.474 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.474 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.474 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.474 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.474 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -4, 2020-09-12T14:04:54.474 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "smithi027~hdd", 2020-09-12T14:04:54.475 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 1, 2020-09-12T14:04:54.475 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "host", 2020-09-12T14:04:54.475 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:04:54.475 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:04:54.475 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:04:54.475 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:04:54.475 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.475 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:04:54.476 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:04:54.476 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:04:54.476 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.476 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.476 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:04:54.476 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:04:54.476 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 1 2020-09-12T14:04:54.476 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.477 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.477 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:04:54.477 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:04:54.477 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 2 2020-09-12T14:04:54.477 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.477 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.477 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.478 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:54.478 INFO:tasks.workunit.client.0.smithi027.stdout: "rules": [ 2020-09-12T14:04:54.478 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.478 INFO:tasks.workunit.client.0.smithi027.stdout: "rule_id": 0, 2020-09-12T14:04:54.478 INFO:tasks.workunit.client.0.smithi027.stdout: "rule_name": "replicated_rule", 2020-09-12T14:04:54.478 INFO:tasks.workunit.client.0.smithi027.stdout: "ruleset": 0, 2020-09-12T14:04:54.478 INFO:tasks.workunit.client.0.smithi027.stdout: "type": 1, 2020-09-12T14:04:54.478 INFO:tasks.workunit.client.0.smithi027.stdout: "min_size": 1, 2020-09-12T14:04:54.479 INFO:tasks.workunit.client.0.smithi027.stdout: "max_size": 10, 2020-09-12T14:04:54.479 INFO:tasks.workunit.client.0.smithi027.stdout: "steps": [ 2020-09-12T14:04:54.479 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.479 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "take", 2020-09-12T14:04:54.479 INFO:tasks.workunit.client.0.smithi027.stdout: "item": -1, 2020-09-12T14:04:54.479 INFO:tasks.workunit.client.0.smithi027.stdout: "item_name": "default" 2020-09-12T14:04:54.479 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.480 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.480 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "choose_firstn", 2020-09-12T14:04:54.480 INFO:tasks.workunit.client.0.smithi027.stdout: "num": 0, 2020-09-12T14:04:54.480 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "osd" 2020-09-12T14:04:54.480 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.480 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.480 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "emit" 2020-09-12T14:04:54.480 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.481 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:04:54.481 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.481 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:54.481 INFO:tasks.workunit.client.0.smithi027.stdout: "tunables": { 2020-09-12T14:04:54.481 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_local_tries": 0, 2020-09-12T14:04:54.481 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_local_fallback_tries": 0, 2020-09-12T14:04:54.481 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_total_tries": 50, 2020-09-12T14:04:54.481 INFO:tasks.workunit.client.0.smithi027.stdout: "chooseleaf_descend_once": 1, 2020-09-12T14:04:54.482 INFO:tasks.workunit.client.0.smithi027.stdout: "chooseleaf_vary_r": 1, 2020-09-12T14:04:54.482 INFO:tasks.workunit.client.0.smithi027.stdout: "chooseleaf_stable": 1, 2020-09-12T14:04:54.482 INFO:tasks.workunit.client.0.smithi027.stdout: "straw_calc_version": 1, 2020-09-12T14:04:54.482 INFO:tasks.workunit.client.0.smithi027.stdout: "allowed_bucket_algs": 54, 2020-09-12T14:04:54.482 INFO:tasks.workunit.client.0.smithi027.stdout: "profile": "jewel", 2020-09-12T14:04:54.482 INFO:tasks.workunit.client.0.smithi027.stdout: "optimal_tunables": 1, 2020-09-12T14:04:54.482 INFO:tasks.workunit.client.0.smithi027.stdout: "legacy_tunables": 0, 2020-09-12T14:04:54.482 INFO:tasks.workunit.client.0.smithi027.stdout: "minimum_required_version": "jewel", 2020-09-12T14:04:54.483 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables": 1, 2020-09-12T14:04:54.483 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables2": 1, 2020-09-12T14:04:54.483 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v2_rules": 0, 2020-09-12T14:04:54.483 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables3": 1, 2020-09-12T14:04:54.483 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v3_rules": 0, 2020-09-12T14:04:54.483 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v4_buckets": 1, 2020-09-12T14:04:54.483 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables5": 1, 2020-09-12T14:04:54.484 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v5_rules": 0 2020-09-12T14:04:54.484 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.484 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_args": {} 2020-09-12T14:04:54.484 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.484 INFO:tasks.workunit.client.0.smithi027.stdout: "fsmap": { 2020-09-12T14:04:54.484 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 1, 2020-09-12T14:04:54.484 INFO:tasks.workunit.client.0.smithi027.stdout: "default_fscid": -1, 2020-09-12T14:04:54.484 INFO:tasks.workunit.client.0.smithi027.stdout: "compat": { 2020-09-12T14:04:54.485 INFO:tasks.workunit.client.0.smithi027.stdout: "compat": {}, 2020-09-12T14:04:54.485 INFO:tasks.workunit.client.0.smithi027.stdout: "ro_compat": {}, 2020-09-12T14:04:54.485 INFO:tasks.workunit.client.0.smithi027.stdout: "incompat": { 2020-09-12T14:04:54.485 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_1": "base v0.20", 2020-09-12T14:04:54.485 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_2": "client writeable ranges", 2020-09-12T14:04:54.485 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_3": "default file layouts on dirs", 2020-09-12T14:04:54.485 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_4": "dir inode in separate object", 2020-09-12T14:04:54.485 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_5": "mds uses versioned encoding", 2020-09-12T14:04:54.486 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_6": "dirfrag is stored in omap", 2020-09-12T14:04:54.486 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_8": "no anchor table", 2020-09-12T14:04:54.486 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_9": "file layout v2", 2020-09-12T14:04:54.486 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_10": "snaprealm v2" 2020-09-12T14:04:54.486 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.486 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.486 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_flags": { 2020-09-12T14:04:54.487 INFO:tasks.workunit.client.0.smithi027.stdout: "enable_multiple": false, 2020-09-12T14:04:54.487 INFO:tasks.workunit.client.0.smithi027.stdout: "ever_enabled_multiple": false 2020-09-12T14:04:54.487 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.487 INFO:tasks.workunit.client.0.smithi027.stdout: "standbys": [], 2020-09-12T14:04:54.487 INFO:tasks.workunit.client.0.smithi027.stdout: "filesystems": [] 2020-09-12T14:04:54.487 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.487 INFO:tasks.workunit.client.0.smithi027.stdout: "mdsmap_first_committed": 1, 2020-09-12T14:04:54.487 INFO:tasks.workunit.client.0.smithi027.stdout: "mdsmap_last_committed": 1, 2020-09-12T14:04:54.488 INFO:tasks.workunit.client.0.smithi027.stdout: "auth": { 2020-09-12T14:04:54.488 INFO:tasks.workunit.client.0.smithi027.stdout: "first_committed": 1, 2020-09-12T14:04:54.488 INFO:tasks.workunit.client.0.smithi027.stdout: "last_committed": 5, 2020-09-12T14:04:54.488 INFO:tasks.workunit.client.0.smithi027.stdout: "num_secrets": 3 2020-09-12T14:04:54.488 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.488 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg": 32, 2020-09-12T14:04:54.488 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_active": 32, 2020-09-12T14:04:54.488 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_unknown": 0, 2020-09-12T14:04:54.489 INFO:tasks.workunit.client.0.smithi027.stdout: "num_osd": 3, 2020-09-12T14:04:54.489 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_sum": { 2020-09-12T14:04:54.489 INFO:tasks.workunit.client.0.smithi027.stdout: "stat_sum": { 2020-09-12T14:04:54.489 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes": 47947, 2020-09-12T14:04:54.489 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects": 100, 2020-09-12T14:04:54.489 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_clones": 0, 2020-09-12T14:04:54.489 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_copies": 300, 2020-09-12T14:04:54.489 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing_on_primary": 0, 2020-09-12T14:04:54.490 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing": 0, 2020-09-12T14:04:54.490 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_degraded": 100, 2020-09-12T14:04:54.490 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_misplaced": 0, 2020-09-12T14:04:54.490 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_unfound": 0, 2020-09-12T14:04:54.490 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_dirty": 100, 2020-09-12T14:04:54.490 INFO:tasks.workunit.client.0.smithi027.stdout: "num_whiteouts": 0, 2020-09-12T14:04:54.490 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read": 0, 2020-09-12T14:04:54.491 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read_kb": 0, 2020-09-12T14:04:54.491 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write": 101, 2020-09-12T14:04:54.491 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write_kb": 101, 2020-09-12T14:04:54.491 INFO:tasks.workunit.client.0.smithi027.stdout: "num_scrub_errors": 0, 2020-09-12T14:04:54.491 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shallow_scrub_errors": 0, 2020-09-12T14:04:54.491 INFO:tasks.workunit.client.0.smithi027.stdout: "num_deep_scrub_errors": 0, 2020-09-12T14:04:54.491 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_recovered": 0, 2020-09-12T14:04:54.491 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_recovered": 0, 2020-09-12T14:04:54.492 INFO:tasks.workunit.client.0.smithi027.stdout: "num_keys_recovered": 0, 2020-09-12T14:04:54.492 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_omap": 0, 2020-09-12T14:04:54.492 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_hit_set_archive": 0, 2020-09-12T14:04:54.492 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_hit_set_archive": 0, 2020-09-12T14:04:54.492 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush": 0, 2020-09-12T14:04:54.492 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_kb": 0, 2020-09-12T14:04:54.492 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict": 0, 2020-09-12T14:04:54.492 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_kb": 0, 2020-09-12T14:04:54.493 INFO:tasks.workunit.client.0.smithi027.stdout: "num_promote": 0, 2020-09-12T14:04:54.493 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_high": 0, 2020-09-12T14:04:54.493 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_low": 0, 2020-09-12T14:04:54.493 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_some": 0, 2020-09-12T14:04:54.493 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_full": 0, 2020-09-12T14:04:54.493 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_pinned": 0, 2020-09-12T14:04:54.493 INFO:tasks.workunit.client.0.smithi027.stdout: "num_legacy_snapsets": 0, 2020-09-12T14:04:54.493 INFO:tasks.workunit.client.0.smithi027.stdout: "num_large_omap_objects": 0, 2020-09-12T14:04:54.494 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_manifest": 0, 2020-09-12T14:04:54.494 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_bytes": 0, 2020-09-12T14:04:54.494 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_keys": 0, 2020-09-12T14:04:54.494 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_repaired": 0 2020-09-12T14:04:54.494 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.494 INFO:tasks.workunit.client.0.smithi027.stdout: "store_stats": { 2020-09-12T14:04:54.494 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 0, 2020-09-12T14:04:54.494 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 0, 2020-09-12T14:04:54.495 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:04:54.495 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 0, 2020-09-12T14:04:54.495 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 0, 2020-09-12T14:04:54.495 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:04:54.495 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:04:54.495 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:04:54.495 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 0, 2020-09-12T14:04:54.496 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 0 2020-09-12T14:04:54.496 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.496 INFO:tasks.workunit.client.0.smithi027.stdout: "log_size": 513, 2020-09-12T14:04:54.496 INFO:tasks.workunit.client.0.smithi027.stdout: "ondisk_log_size": 513, 2020-09-12T14:04:54.496 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 64, 2020-09-12T14:04:54.496 INFO:tasks.workunit.client.0.smithi027.stdout: "acting": 64, 2020-09-12T14:04:54.496 INFO:tasks.workunit.client.0.smithi027.stdout: "num_store_stats": 0 2020-09-12T14:04:54.496 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.497 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_sum": { 2020-09-12T14:04:54.497 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 0, 2020-09-12T14:04:54.497 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 0, 2020-09-12T14:04:54.497 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pgs": 65, 2020-09-12T14:04:54.497 INFO:tasks.workunit.client.0.smithi027.stdout: "num_osds": 3, 2020-09-12T14:04:54.497 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_osds": 3, 2020-09-12T14:04:54.497 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_omap_osds": 3, 2020-09-12T14:04:54.497 INFO:tasks.workunit.client.0.smithi027.stdout: "kb": 314572800, 2020-09-12T14:04:54.498 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used": 19596, 2020-09-12T14:04:54.498 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_data": 2220, 2020-09-12T14:04:54.498 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_omap": 1, 2020-09-12T14:04:54.498 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_meta": 17342, 2020-09-12T14:04:54.498 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_avail": 314553204, 2020-09-12T14:04:54.498 INFO:tasks.workunit.client.0.smithi027.stdout: "statfs": { 2020-09-12T14:04:54.498 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 322122547200, 2020-09-12T14:04:54.498 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 322102480896, 2020-09-12T14:04:54.499 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:04:54.499 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 2273280, 2020-09-12T14:04:54.499 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 464868, 2020-09-12T14:04:54.499 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:04:54.499 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:04:54.499 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:04:54.499 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 1928, 2020-09-12T14:04:54.500 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 17758328 2020-09-12T14:04:54.500 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.500 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_peers": [], 2020-09-12T14:04:54.500 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_trim_queue_len": 0, 2020-09-12T14:04:54.500 INFO:tasks.workunit.client.0.smithi027.stdout: "num_snap_trimming": 0, 2020-09-12T14:04:54.500 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shards_repaired": 0, 2020-09-12T14:04:54.500 INFO:tasks.workunit.client.0.smithi027.stdout: "op_queue_age_hist": { 2020-09-12T14:04:54.500 INFO:tasks.workunit.client.0.smithi027.stdout: "histogram": [], 2020-09-12T14:04:54.501 INFO:tasks.workunit.client.0.smithi027.stdout: "upper_bound": 1 2020-09-12T14:04:54.501 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.501 INFO:tasks.workunit.client.0.smithi027.stdout: "perf_stat": { 2020-09-12T14:04:54.501 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ms": 43, 2020-09-12T14:04:54.501 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ms": 43, 2020-09-12T14:04:54.501 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ns": 43000000, 2020-09-12T14:04:54.501 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ns": 43000000 2020-09-12T14:04:54.501 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.502 INFO:tasks.workunit.client.0.smithi027.stdout: "alerts": [], 2020-09-12T14:04:54.502 INFO:tasks.workunit.client.0.smithi027.stdout: "network_ping_times": [] 2020-09-12T14:04:54.502 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.502 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_sum_by_class": { 2020-09-12T14:04:54.502 INFO:tasks.workunit.client.0.smithi027.stdout: "hdd": { 2020-09-12T14:04:54.502 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 0, 2020-09-12T14:04:54.502 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 0, 2020-09-12T14:04:54.502 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pgs": 65, 2020-09-12T14:04:54.503 INFO:tasks.workunit.client.0.smithi027.stdout: "num_osds": 3, 2020-09-12T14:04:54.503 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_osds": 3, 2020-09-12T14:04:54.503 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_omap_osds": 3, 2020-09-12T14:04:54.503 INFO:tasks.workunit.client.0.smithi027.stdout: "kb": 314572800, 2020-09-12T14:04:54.503 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used": 19596, 2020-09-12T14:04:54.503 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_data": 2220, 2020-09-12T14:04:54.503 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_omap": 1, 2020-09-12T14:04:54.503 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_meta": 17342, 2020-09-12T14:04:54.504 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_avail": 314553204, 2020-09-12T14:04:54.504 INFO:tasks.workunit.client.0.smithi027.stdout: "statfs": { 2020-09-12T14:04:54.504 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 322122547200, 2020-09-12T14:04:54.504 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 322102480896, 2020-09-12T14:04:54.504 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:04:54.504 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 2273280, 2020-09-12T14:04:54.504 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 464868, 2020-09-12T14:04:54.505 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:04:54.505 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:04:54.505 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:04:54.505 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 1928, 2020-09-12T14:04:54.505 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 17758328 2020-09-12T14:04:54.505 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.505 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_peers": [], 2020-09-12T14:04:54.505 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_trim_queue_len": 0, 2020-09-12T14:04:54.506 INFO:tasks.workunit.client.0.smithi027.stdout: "num_snap_trimming": 0, 2020-09-12T14:04:54.506 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shards_repaired": 0, 2020-09-12T14:04:54.506 INFO:tasks.workunit.client.0.smithi027.stdout: "op_queue_age_hist": { 2020-09-12T14:04:54.506 INFO:tasks.workunit.client.0.smithi027.stdout: "histogram": [], 2020-09-12T14:04:54.506 INFO:tasks.workunit.client.0.smithi027.stdout: "upper_bound": 1 2020-09-12T14:04:54.506 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.506 INFO:tasks.workunit.client.0.smithi027.stdout: "perf_stat": { 2020-09-12T14:04:54.506 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ms": 43, 2020-09-12T14:04:54.507 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ms": 43, 2020-09-12T14:04:54.507 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ns": 43000000, 2020-09-12T14:04:54.507 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ns": 43000000 2020-09-12T14:04:54.507 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.507 INFO:tasks.workunit.client.0.smithi027.stdout: "alerts": [], 2020-09-12T14:04:54.507 INFO:tasks.workunit.client.0.smithi027.stdout: "network_ping_times": [] 2020-09-12T14:04:54.507 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.508 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.508 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_stats": [ 2020-09-12T14:04:54.508 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.508 INFO:tasks.workunit.client.0.smithi027.stdout: "poolid": 1, 2020-09-12T14:04:54.508 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg": 32, 2020-09-12T14:04:54.508 INFO:tasks.workunit.client.0.smithi027.stdout: "stat_sum": { 2020-09-12T14:04:54.508 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes": 47947, 2020-09-12T14:04:54.508 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects": 100, 2020-09-12T14:04:54.509 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_clones": 0, 2020-09-12T14:04:54.509 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_copies": 300, 2020-09-12T14:04:54.509 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing_on_primary": 0, 2020-09-12T14:04:54.509 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing": 0, 2020-09-12T14:04:54.509 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_degraded": 100, 2020-09-12T14:04:54.509 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_misplaced": 0, 2020-09-12T14:04:54.509 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_unfound": 0, 2020-09-12T14:04:54.509 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_dirty": 100, 2020-09-12T14:04:54.510 INFO:tasks.workunit.client.0.smithi027.stdout: "num_whiteouts": 0, 2020-09-12T14:04:54.510 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read": 0, 2020-09-12T14:04:54.510 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read_kb": 0, 2020-09-12T14:04:54.510 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write": 101, 2020-09-12T14:04:54.510 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write_kb": 101, 2020-09-12T14:04:54.510 INFO:tasks.workunit.client.0.smithi027.stdout: "num_scrub_errors": 0, 2020-09-12T14:04:54.510 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shallow_scrub_errors": 0, 2020-09-12T14:04:54.510 INFO:tasks.workunit.client.0.smithi027.stdout: "num_deep_scrub_errors": 0, 2020-09-12T14:04:54.511 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_recovered": 0, 2020-09-12T14:04:54.511 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_recovered": 0, 2020-09-12T14:04:54.511 INFO:tasks.workunit.client.0.smithi027.stdout: "num_keys_recovered": 0, 2020-09-12T14:04:54.511 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_omap": 0, 2020-09-12T14:04:54.511 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_hit_set_archive": 0, 2020-09-12T14:04:54.511 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_hit_set_archive": 0, 2020-09-12T14:04:54.511 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush": 0, 2020-09-12T14:04:54.512 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_kb": 0, 2020-09-12T14:04:54.512 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict": 0, 2020-09-12T14:04:54.512 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_kb": 0, 2020-09-12T14:04:54.512 INFO:tasks.workunit.client.0.smithi027.stdout: "num_promote": 0, 2020-09-12T14:04:54.512 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_high": 0, 2020-09-12T14:04:54.512 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_low": 0, 2020-09-12T14:04:54.512 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_some": 0, 2020-09-12T14:04:54.512 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_full": 0, 2020-09-12T14:04:54.513 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_pinned": 0, 2020-09-12T14:04:54.513 INFO:tasks.workunit.client.0.smithi027.stdout: "num_legacy_snapsets": 0, 2020-09-12T14:04:54.513 INFO:tasks.workunit.client.0.smithi027.stdout: "num_large_omap_objects": 0, 2020-09-12T14:04:54.513 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_manifest": 0, 2020-09-12T14:04:54.513 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_bytes": 0, 2020-09-12T14:04:54.513 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_keys": 0, 2020-09-12T14:04:54.513 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_repaired": 0 2020-09-12T14:04:54.513 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.514 INFO:tasks.workunit.client.0.smithi027.stdout: "store_stats": { 2020-09-12T14:04:54.514 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 0, 2020-09-12T14:04:54.514 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 0, 2020-09-12T14:04:54.514 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:04:54.514 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 1220608, 2020-09-12T14:04:54.514 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 142522, 2020-09-12T14:04:54.514 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:04:54.514 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:04:54.515 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:04:54.515 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 0, 2020-09-12T14:04:54.515 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 0 2020-09-12T14:04:54.515 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.515 INFO:tasks.workunit.client.0.smithi027.stdout: "log_size": 513, 2020-09-12T14:04:54.515 INFO:tasks.workunit.client.0.smithi027.stdout: "ondisk_log_size": 513, 2020-09-12T14:04:54.515 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 64, 2020-09-12T14:04:54.515 INFO:tasks.workunit.client.0.smithi027.stdout: "acting": 64, 2020-09-12T14:04:54.516 INFO:tasks.workunit.client.0.smithi027.stdout: "num_store_stats": 3 2020-09-12T14:04:54.516 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.516 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:54.516 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_stats": [ 2020-09-12T14:04:54.516 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.516 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:04:54.516 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 171798691860 2020-09-12T14:04:54.517 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.517 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.517 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:04:54.517 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 42949672977 2020-09-12T14:04:54.517 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.517 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.517 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:04:54.517 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 176093659156 2020-09-12T14:04:54.518 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.518 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:54.518 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_by_state": [ 2020-09-12T14:04:54.518 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.518 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "active+undersized+degraded", 2020-09-12T14:04:54.518 INFO:tasks.workunit.client.0.smithi027.stdout: "num": 32 2020-09-12T14:04:54.518 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.518 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:54.519 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_by_osd": [ 2020-09-12T14:04:54.519 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.519 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:04:54.519 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 0, 2020-09-12T14:04:54.519 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 32, 2020-09-12T14:04:54.519 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T14:04:54.519 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.519 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.520 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:04:54.520 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 0, 2020-09-12T14:04:54.520 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 0, 2020-09-12T14:04:54.520 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T14:04:54.520 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.520 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.520 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:04:54.520 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 32, 2020-09-12T14:04:54.521 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 32, 2020-09-12T14:04:54.521 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T14:04:54.521 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.521 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:54.521 INFO:tasks.workunit.client.0.smithi027.stdout: "purged_snaps": [ 2020-09-12T14:04:54.521 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:04:54.521 INFO:tasks.workunit.client.0.smithi027.stdout: "pool": 1, 2020-09-12T14:04:54.522 INFO:tasks.workunit.client.0.smithi027.stdout: "purged_snaps": {} 2020-09-12T14:04:54.522 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.522 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:04:54.522 INFO:tasks.workunit.client.0.smithi027.stdout: "servicemap": { 2020-09-12T14:04:54.522 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 6, 2020-09-12T14:04:54.522 INFO:tasks.workunit.client.0.smithi027.stdout: "modified": "2020-09-12T14:04:38.654229+0000", 2020-09-12T14:04:54.522 INFO:tasks.workunit.client.0.smithi027.stdout: "services": {} 2020-09-12T14:04:54.522 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:04:54.523 INFO:tasks.workunit.client.0.smithi027.stdout: "paxos": { 2020-09-12T14:04:54.523 INFO:tasks.workunit.client.0.smithi027.stdout: "first_committed": 1, 2020-09-12T14:04:54.523 INFO:tasks.workunit.client.0.smithi027.stdout: "last_committed": 470, 2020-09-12T14:04:54.523 INFO:tasks.workunit.client.0.smithi027.stdout: "last_pn": 0, 2020-09-12T14:04:54.523 INFO:tasks.workunit.client.0.smithi027.stdout: "accepted_pn": 0 2020-09-12T14:04:54.523 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:04:54.523 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:04:54.524 INFO:tasks.workunit.client.0.smithi027.stderr:report 1262833543 2020-09-12T14:04:54.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1604: wait_for_clean: return 1 2020-09-12T14:04:54.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:746: TEST_divergent_3: echo 'delay recovery' 2020-09-12T14:04:54.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:747: TEST_divergent_3: ceph pg dump pgs 2020-09-12T14:04:54.525 INFO:tasks.workunit.client.0.smithi027.stdout:delay recovery 2020-09-12T14:04:54.756 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:04:54.757 INFO:tasks.workunit.client.0.smithi027.stdout:1.1f 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958633+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.757 INFO:tasks.workunit.client.0.smithi027.stdout:1.1e 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958019+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.757 INFO:tasks.workunit.client.0.smithi027.stdout:1.1d 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958447+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.757 INFO:tasks.workunit.client.0.smithi027.stdout:1.1c 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958753+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.757 INFO:tasks.workunit.client.0.smithi027.stdout:1.1b 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958199+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.757 INFO:tasks.workunit.client.0.smithi027.stdout:1.1a 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955975+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.757 INFO:tasks.workunit.client.0.smithi027.stdout:1.19 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.813511+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.758 INFO:tasks.workunit.client.0.smithi027.stdout:1.18 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958032+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.758 INFO:tasks.workunit.client.0.smithi027.stdout:1.17 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958327+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.758 INFO:tasks.workunit.client.0.smithi027.stdout:1.16 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957794+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.758 INFO:tasks.workunit.client.0.smithi027.stdout:1.15 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957055+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.758 INFO:tasks.workunit.client.0.smithi027.stdout:1.14 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955987+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.758 INFO:tasks.workunit.client.0.smithi027.stdout:1.13 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957641+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.758 INFO:tasks.workunit.client.0.smithi027.stdout:1.12 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957558+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.759 INFO:tasks.workunit.client.0.smithi027.stdout:1.11 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957392+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.759 INFO:tasks.workunit.client.0.smithi027.stdout:1.10 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957442+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.759 INFO:tasks.workunit.client.0.smithi027.stdout:1.5 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957006+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.759 INFO:tasks.workunit.client.0.smithi027.stdout:1.4 4 0 4 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957032+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.759 INFO:tasks.workunit.client.0.smithi027.stdout:1.3 4 0 4 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.956677+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.759 INFO:tasks.workunit.client.0.smithi027.stdout:1.2 4 0 4 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.956981+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.759 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958434+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.760 INFO:tasks.workunit.client.0.smithi027.stdout:1.1 4 0 4 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957840+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.760 INFO:tasks.workunit.client.0.smithi027.stdout:1.6 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958222+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.760 INFO:tasks.workunit.client.0.smithi027.stdout:1.7 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955938+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.760 INFO:tasks.workunit.client.0.smithi027.stdout:1.8 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955987+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.760 INFO:tasks.workunit.client.0.smithi027.stdout:1.9 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955983+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.760 INFO:tasks.workunit.client.0.smithi027.stdout:1.a 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958911+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.760 INFO:tasks.workunit.client.0.smithi027.stdout:1.b 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.956643+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.760 INFO:tasks.workunit.client.0.smithi027.stdout:1.c 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.956672+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.761 INFO:tasks.workunit.client.0.smithi027.stdout:1.d 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957160+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.761 INFO:tasks.workunit.client.0.smithi027.stdout:1.e 3 0 3 0 0 1156 0 0 17 17 active+undersized+degraded 2020-09-12T14:03:39.956415+0000 47'101 47:127 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.761 INFO:tasks.workunit.client.0.smithi027.stdout:1.f 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957591+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:54.761 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:04:54.761 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:04:54.762 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:04:54.767 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:748: TEST_divergent_3: for i in $non_divergent 2020-09-12T14:04:54.767 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:750: TEST_divergent_3: get_asok_path osd.0 2020-09-12T14:04:54.768 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2020-09-12T14:04:54.768 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2020-09-12T14:04:54.768 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:04:54.769 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:04:54.769 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:04:54.769 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.0.asok 2020-09-12T14:04:54.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:750: TEST_divergent_3: CEPH_ARGS= 2020-09-12T14:04:54.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:750: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.0.asok set_recovery_delay 100000 2020-09-12T14:04:54.875 INFO:tasks.workunit.client.0.smithi027.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2020-09-12T14:04:54.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:748: TEST_divergent_3: for i in $non_divergent 2020-09-12T14:04:54.884 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:750: TEST_divergent_3: get_asok_path osd.2 2020-09-12T14:04:54.884 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2020-09-12T14:04:54.884 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2020-09-12T14:04:54.884 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:04:54.885 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:04:54.885 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:04:54.885 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.2.asok 2020-09-12T14:04:54.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:750: TEST_divergent_3: CEPH_ARGS= 2020-09-12T14:04:54.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:750: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.2.asok set_recovery_delay 100000 2020-09-12T14:04:54.991 INFO:tasks.workunit.client.0.smithi027.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2020-09-12T14:04:55.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:754: TEST_divergent_3: echo 'revive divergent 1' 2020-09-12T14:04:55.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:755: TEST_divergent_3: ceph pg dump pgs 2020-09-12T14:04:55.000 INFO:tasks.workunit.client.0.smithi027.stdout:revive divergent 1 2020-09-12T14:04:55.386 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:04:55.387 INFO:tasks.workunit.client.0.smithi027.stdout:1.1f 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958633+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.387 INFO:tasks.workunit.client.0.smithi027.stdout:1.1e 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958019+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.387 INFO:tasks.workunit.client.0.smithi027.stdout:1.1d 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958447+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.387 INFO:tasks.workunit.client.0.smithi027.stdout:1.1c 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958753+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.387 INFO:tasks.workunit.client.0.smithi027.stdout:1.1b 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958199+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.387 INFO:tasks.workunit.client.0.smithi027.stdout:1.1a 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955975+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.388 INFO:tasks.workunit.client.0.smithi027.stdout:1.19 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.813511+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.388 INFO:tasks.workunit.client.0.smithi027.stdout:1.18 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958032+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.388 INFO:tasks.workunit.client.0.smithi027.stdout:1.17 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958327+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.388 INFO:tasks.workunit.client.0.smithi027.stdout:1.16 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957794+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.388 INFO:tasks.workunit.client.0.smithi027.stdout:1.15 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957055+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.388 INFO:tasks.workunit.client.0.smithi027.stdout:1.14 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955987+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.388 INFO:tasks.workunit.client.0.smithi027.stdout:1.13 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957641+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.388 INFO:tasks.workunit.client.0.smithi027.stdout:1.12 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957558+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.389 INFO:tasks.workunit.client.0.smithi027.stdout:1.11 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957392+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.389 INFO:tasks.workunit.client.0.smithi027.stdout:1.10 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957442+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.389 INFO:tasks.workunit.client.0.smithi027.stdout:1.5 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957006+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.389 INFO:tasks.workunit.client.0.smithi027.stdout:1.4 4 0 4 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957032+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.389 INFO:tasks.workunit.client.0.smithi027.stdout:1.3 4 0 4 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.956677+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.389 INFO:tasks.workunit.client.0.smithi027.stdout:1.2 4 0 4 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.956981+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.389 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958434+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.390 INFO:tasks.workunit.client.0.smithi027.stdout:1.1 4 0 4 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957840+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.390 INFO:tasks.workunit.client.0.smithi027.stdout:1.6 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958222+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.390 INFO:tasks.workunit.client.0.smithi027.stdout:1.7 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955938+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.390 INFO:tasks.workunit.client.0.smithi027.stdout:1.8 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955987+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.390 INFO:tasks.workunit.client.0.smithi027.stdout:1.9 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955983+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.390 INFO:tasks.workunit.client.0.smithi027.stdout:1.a 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958911+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.390 INFO:tasks.workunit.client.0.smithi027.stdout:1.b 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.956643+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.391 INFO:tasks.workunit.client.0.smithi027.stdout:1.c 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.956672+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.391 INFO:tasks.workunit.client.0.smithi027.stdout:1.d 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957160+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.391 INFO:tasks.workunit.client.0.smithi027.stdout:1.e 3 0 3 0 0 1156 0 0 17 17 active+undersized+degraded 2020-09-12T14:03:39.956415+0000 47'101 47:127 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.391 INFO:tasks.workunit.client.0.smithi027.stdout:1.f 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957591+0000 31'100 47:126 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:04:55.391 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:04:55.391 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:04:55.392 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:04:55.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:756: TEST_divergent_3: ceph osd set noup 2020-09-12T14:04:56.191 INFO:tasks.workunit.client.0.smithi027.stderr:noup is set 2020-09-12T14:04:56.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:757: TEST_divergent_3: activate_osd td/divergent-priors 1 2020-09-12T14:04:56.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/divergent-priors 2020-09-12T14:04:56.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T14:04:56.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=1 2020-09-12T14:04:56.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T14:04:56.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/divergent-priors/1 2020-09-12T14:04:56.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T14:04:56.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:04:56.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:04:56.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:04:56.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2020-09-12T14:04:56.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2020-09-12T14:04:56.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T14:04:56.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T14:04:56.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T14:04:56.205 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T14:04:56.205 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:04:56.206 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:04:56.206 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:04:56.206 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:04:56.206 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:04:56.206 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:04:56.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:04:56.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:04:56.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T14:04:56.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T14:04:56.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:04:56.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:04:56.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:04:56.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T14:04:56.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T14:04:56.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/divergent-priors/1 2020-09-12T14:04:56.208 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T14:04:56.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.1 2020-09-12T14:04:56.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 1 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:04:56.209 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/divergent-priors/1/whoami 2020-09-12T14:04:56.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 1 = 1 ']' 2020-09-12T14:04:56.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T14:04:56.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T14:04:56.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T14:04:56.233 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:04:56.232+0000 7f44e07deec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:04:56.238 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:04:56.237+0000 7f44e07deec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:04:56.239 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:04:56.238+0000 7f44e07deec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:04:56.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:877: activate_osd: return 0 2020-09-12T14:04:56.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:758: TEST_divergent_3: sleep 5 2020-09-12T14:04:56.755 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:04:56.755+0000 7f44e07deec0 -1 Falling back to public interface 2020-09-12T14:04:58.548 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:04:58.547+0000 7f44e07deec0 -1 osd.1 34 log_to_monitors {default=true} 2020-09-12T14:05:01.620 INFO:tasks.workunit.client.0.smithi027.stdout:delay recovery divergent 2020-09-12T14:05:01.621 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:760: TEST_divergent_3: echo 'delay recovery divergent' 2020-09-12T14:05:01.621 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:761: TEST_divergent_3: ceph pg dump pgs 2020-09-12T14:05:02.015 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:05:02.016 INFO:tasks.workunit.client.0.smithi027.stdout:1.1f 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958633+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.016 INFO:tasks.workunit.client.0.smithi027.stdout:1.1e 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958019+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.016 INFO:tasks.workunit.client.0.smithi027.stdout:1.1d 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958447+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.016 INFO:tasks.workunit.client.0.smithi027.stdout:1.1c 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958753+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.016 INFO:tasks.workunit.client.0.smithi027.stdout:1.1b 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958199+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.017 INFO:tasks.workunit.client.0.smithi027.stdout:1.1a 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955975+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.017 INFO:tasks.workunit.client.0.smithi027.stdout:1.19 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.813511+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.017 INFO:tasks.workunit.client.0.smithi027.stdout:1.18 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958032+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.017 INFO:tasks.workunit.client.0.smithi027.stdout:1.17 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958327+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.017 INFO:tasks.workunit.client.0.smithi027.stdout:1.16 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957794+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.017 INFO:tasks.workunit.client.0.smithi027.stdout:1.15 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957055+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.017 INFO:tasks.workunit.client.0.smithi027.stdout:1.14 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955987+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.018 INFO:tasks.workunit.client.0.smithi027.stdout:1.13 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957641+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.018 INFO:tasks.workunit.client.0.smithi027.stdout:1.12 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957558+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.018 INFO:tasks.workunit.client.0.smithi027.stdout:1.11 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957392+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.018 INFO:tasks.workunit.client.0.smithi027.stdout:1.10 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957442+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.018 INFO:tasks.workunit.client.0.smithi027.stdout:1.5 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957006+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.018 INFO:tasks.workunit.client.0.smithi027.stdout:1.4 4 0 4 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957032+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.018 INFO:tasks.workunit.client.0.smithi027.stdout:1.3 4 0 4 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.956677+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.019 INFO:tasks.workunit.client.0.smithi027.stdout:1.2 4 0 4 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.956981+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.019 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958434+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.019 INFO:tasks.workunit.client.0.smithi027.stdout:1.1 4 0 4 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957840+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.019 INFO:tasks.workunit.client.0.smithi027.stdout:1.6 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958222+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.019 INFO:tasks.workunit.client.0.smithi027.stdout:1.7 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955938+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.019 INFO:tasks.workunit.client.0.smithi027.stdout:1.8 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955987+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.019 INFO:tasks.workunit.client.0.smithi027.stdout:1.9 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955983+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.020 INFO:tasks.workunit.client.0.smithi027.stdout:1.a 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958911+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.020 INFO:tasks.workunit.client.0.smithi027.stdout:1.b 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.956643+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.020 INFO:tasks.workunit.client.0.smithi027.stdout:1.c 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.956672+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.020 INFO:tasks.workunit.client.0.smithi027.stdout:1.d 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957160+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.020 INFO:tasks.workunit.client.0.smithi027.stdout:1.e 3 0 3 0 0 1156 0 0 17 17 active+undersized+degraded 2020-09-12T14:03:39.956415+0000 47'101 48:129 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.020 INFO:tasks.workunit.client.0.smithi027.stdout:1.f 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957591+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:02.020 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:05:02.020 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:05:02.021 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:05:02.027 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:762: TEST_divergent_3: get_asok_path osd.1 2020-09-12T14:05:02.027 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2020-09-12T14:05:02.027 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2020-09-12T14:05:02.027 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:05:02.027 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:05:02.028 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:05:02.028 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.1.asok 2020-09-12T14:05:02.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:762: TEST_divergent_3: CEPH_ARGS= 2020-09-12T14:05:02.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:762: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.1.asok set_recovery_delay 100000 2020-09-12T14:05:02.135 INFO:tasks.workunit.client.0.smithi027.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2020-09-12T14:05:02.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:764: TEST_divergent_3: ceph osd unset noup 2020-09-12T14:05:02.880 INFO:tasks.workunit.client.0.smithi027.stderr:noup is unset 2020-09-12T14:05:02.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:766: TEST_divergent_3: wait_for_osd up 0 2020-09-12T14:05:02.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:05:02.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T14:05:02.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:05:02.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:05:02.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:05:02.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:05:02.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:05:02.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:05:02.890 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:05:03.331 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 40 up_thru 45 down_at 35 last_clean_interval [5,31) [v2:127.0.0.1:6802/49081,v1:127.0.0.1:6803/49081] [v2:127.0.0.1:6804/49081,v1:127.0.0.1:6805/49081] exists,up 9d8d2ee9-0710-4a4e-a0ba-cec45cbf29a2 2020-09-12T14:05:03.332 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:05:03.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:05:03.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:05:03.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:05:03.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:767: TEST_divergent_3: wait_for_osd up 1 2020-09-12T14:05:03.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:05:03.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:05:03.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:05:03.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:05:03.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:05:03.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:05:03.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:05:03.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:05:03.748 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 51 up_thru 51 down_at 36 last_clean_interval [10,34) [v2:127.0.0.1:6818/52023,v1:127.0.0.1:6819/52023] [v2:127.0.0.1:6820/52023,v1:127.0.0.1:6821/52023] exists,up 06975646-fe2b-4ca5-82e0-a2d0e01e06a8 2020-09-12T14:05:03.748 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:05:03.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:05:03.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:05:03.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:05:03.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:768: TEST_divergent_3: wait_for_osd up 2 2020-09-12T14:05:03.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:05:03.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T14:05:03.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:05:03.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:05:03.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:05:03.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:05:03.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:05:03.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:05:04.161 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up in weight 1 up_from 41 up_thru 0 down_at 37 last_clean_interval [15,31) [v2:127.0.0.1:6810/49480,v1:127.0.0.1:6811/49480] [v2:127.0.0.1:6812/49480,v1:127.0.0.1:6813/49480] exists,up ed2b43d9-2a50-45c4-b6a7-8f00f86304e8 2020-09-12T14:05:04.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:05:04.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:05:04.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:05:04.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:770: TEST_divergent_3: ceph pg dump pgs 2020-09-12T14:05:04.584 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:05:04.584 INFO:tasks.workunit.client.0.smithi027.stdout:1.1f 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958633+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.584 INFO:tasks.workunit.client.0.smithi027.stdout:1.1e 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958019+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.584 INFO:tasks.workunit.client.0.smithi027.stdout:1.1d 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958447+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.584 INFO:tasks.workunit.client.0.smithi027.stdout:1.1c 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958753+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.585 INFO:tasks.workunit.client.0.smithi027.stdout:1.1b 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958199+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.585 INFO:tasks.workunit.client.0.smithi027.stdout:1.1a 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955975+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.585 INFO:tasks.workunit.client.0.smithi027.stdout:1.19 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.813511+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.585 INFO:tasks.workunit.client.0.smithi027.stdout:1.18 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958032+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.585 INFO:tasks.workunit.client.0.smithi027.stdout:1.17 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958327+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.585 INFO:tasks.workunit.client.0.smithi027.stdout:1.16 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957794+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.585 INFO:tasks.workunit.client.0.smithi027.stdout:1.15 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957055+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.586 INFO:tasks.workunit.client.0.smithi027.stdout:1.14 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955987+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.586 INFO:tasks.workunit.client.0.smithi027.stdout:1.13 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957641+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.586 INFO:tasks.workunit.client.0.smithi027.stdout:1.12 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957558+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.586 INFO:tasks.workunit.client.0.smithi027.stdout:1.11 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957392+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.586 INFO:tasks.workunit.client.0.smithi027.stdout:1.10 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957442+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.586 INFO:tasks.workunit.client.0.smithi027.stdout:1.5 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957006+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.586 INFO:tasks.workunit.client.0.smithi027.stdout:1.4 4 0 4 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957032+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.587 INFO:tasks.workunit.client.0.smithi027.stdout:1.3 4 0 4 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.956677+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.587 INFO:tasks.workunit.client.0.smithi027.stdout:1.2 4 0 4 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.956981+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.587 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958434+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.587 INFO:tasks.workunit.client.0.smithi027.stdout:1.1 4 0 4 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957840+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.587 INFO:tasks.workunit.client.0.smithi027.stdout:1.6 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958222+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.587 INFO:tasks.workunit.client.0.smithi027.stdout:1.7 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955938+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.587 INFO:tasks.workunit.client.0.smithi027.stdout:1.8 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955987+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.588 INFO:tasks.workunit.client.0.smithi027.stdout:1.9 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.955983+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.588 INFO:tasks.workunit.client.0.smithi027.stdout:1.a 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.958911+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.588 INFO:tasks.workunit.client.0.smithi027.stdout:1.b 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.956643+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.588 INFO:tasks.workunit.client.0.smithi027.stdout:1.c 3 0 3 0 0 1510 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.956672+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.588 INFO:tasks.workunit.client.0.smithi027.stdout:1.d 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957160+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.588 INFO:tasks.workunit.client.0.smithi027.stdout:1.e 3 0 3 0 0 1156 0 0 17 17 active+undersized+degraded 2020-09-12T14:03:39.956415+0000 47'101 48:129 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.588 INFO:tasks.workunit.client.0.smithi027.stdout:1.f 3 0 3 0 0 1509 0 0 16 16 active+undersized+degraded 2020-09-12T14:03:39.957591+0000 31'100 48:128 [0,2] 0 [0,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.588 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:05:04.589 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:05:04.589 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:05:04.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:771: TEST_divergent_3: echo 'wait for peering' 2020-09-12T14:05:04.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:772: TEST_divergent_3: ceph pg dump pgs 2020-09-12T14:05:04.595 INFO:tasks.workunit.client.0.smithi027.stdout:wait for peering 2020-09-12T14:05:04.980 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:05:04.980 INFO:tasks.workunit.client.0.smithi027.stdout:1.1f 3 0 0 0 0 1509 0 0 16 16 activating 2020-09-12T14:05:03.479567+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.981 INFO:tasks.workunit.client.0.smithi027.stdout:1.1e 3 0 0 0 0 1509 0 0 10 10 activating 2020-09-12T14:05:03.481535+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.981 INFO:tasks.workunit.client.0.smithi027.stdout:1.1d 3 0 0 0 0 1509 0 0 16 16 activating 2020-09-12T14:05:03.481793+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.981 INFO:tasks.workunit.client.0.smithi027.stdout:1.1c 3 0 0 0 0 1509 0 0 16 16 activating 2020-09-12T14:05:03.481481+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.981 INFO:tasks.workunit.client.0.smithi027.stdout:1.1b 3 1 1 0 0 1509 0 0 10 10 activating 2020-09-12T14:05:03.481784+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.981 INFO:tasks.workunit.client.0.smithi027.stdout:1.1a 3 0 0 0 0 1509 0 0 16 16 activating 2020-09-12T14:05:03.481763+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.981 INFO:tasks.workunit.client.0.smithi027.stdout:1.19 3 0 0 0 0 1509 0 0 16 16 activating 2020-09-12T14:05:03.484569+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.981 INFO:tasks.workunit.client.0.smithi027.stdout:1.18 3 0 0 0 0 1509 0 0 10 10 activating 2020-09-12T14:05:03.480775+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.982 INFO:tasks.workunit.client.0.smithi027.stdout:1.17 3 0 0 0 0 1509 0 0 16 16 activating 2020-09-12T14:05:03.480962+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.982 INFO:tasks.workunit.client.0.smithi027.stdout:1.16 3 0 0 0 0 1509 0 0 16 16 activating 2020-09-12T14:05:03.481149+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.982 INFO:tasks.workunit.client.0.smithi027.stdout:1.15 3 0 0 0 0 1509 0 0 16 16 activating 2020-09-12T14:05:03.479000+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.982 INFO:tasks.workunit.client.0.smithi027.stdout:1.14 3 0 0 0 0 1509 0 0 10 10 activating 2020-09-12T14:05:03.483563+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.982 INFO:tasks.workunit.client.0.smithi027.stdout:1.13 3 0 0 0 0 1509 0 0 10 10 activating 2020-09-12T14:05:03.481293+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.982 INFO:tasks.workunit.client.0.smithi027.stdout:1.12 3 0 0 0 0 1509 0 0 16 16 activating 2020-09-12T14:05:03.480454+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.983 INFO:tasks.workunit.client.0.smithi027.stdout:1.11 3 1 1 0 0 1509 0 0 16 16 activating 2020-09-12T14:05:03.480619+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.983 INFO:tasks.workunit.client.0.smithi027.stdout:1.10 3 0 0 0 0 1509 0 0 10 10 activating 2020-09-12T14:05:03.480128+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.983 INFO:tasks.workunit.client.0.smithi027.stdout:1.5 3 0 0 0 0 1510 0 0 10 10 activating 2020-09-12T14:05:03.483103+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.983 INFO:tasks.workunit.client.0.smithi027.stdout:1.4 4 0 0 0 0 1510 0 0 16 16 activating 2020-09-12T14:05:03.480249+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.983 INFO:tasks.workunit.client.0.smithi027.stdout:1.3 4 0 0 0 0 1510 0 0 10 10 activating 2020-09-12T14:05:03.479570+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.983 INFO:tasks.workunit.client.0.smithi027.stdout:1.2 4 0 0 0 0 1510 0 0 16 16 activating 2020-09-12T14:05:03.480035+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.983 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 3 0 0 0 0 1509 0 0 10 10 activating 2020-09-12T14:05:03.478943+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.983 INFO:tasks.workunit.client.0.smithi027.stdout:1.1 4 0 0 0 0 1510 0 0 15 15 activating 2020-09-12T14:05:03.480676+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.984 INFO:tasks.workunit.client.0.smithi027.stdout:1.6 3 0 0 0 0 1510 0 0 16 16 activating 2020-09-12T14:05:03.482312+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.984 INFO:tasks.workunit.client.0.smithi027.stdout:1.7 3 0 0 0 0 1510 0 0 16 16 activating 2020-09-12T14:05:03.479005+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.984 INFO:tasks.workunit.client.0.smithi027.stdout:1.8 3 0 0 0 0 1510 0 0 10 10 activating 2020-09-12T14:05:03.479031+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.984 INFO:tasks.workunit.client.0.smithi027.stdout:1.9 3 1 1 0 0 1510 0 0 11 11 activating 2020-09-12T14:05:03.479145+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.984 INFO:tasks.workunit.client.0.smithi027.stdout:1.a 3 0 0 0 0 1510 0 0 16 16 activating 2020-09-12T14:05:03.484067+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.984 INFO:tasks.workunit.client.0.smithi027.stdout:1.b 3 0 0 0 0 1510 0 0 10 10 activating 2020-09-12T14:05:03.481212+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.984 INFO:tasks.workunit.client.0.smithi027.stdout:1.c 3 0 0 0 0 1510 0 0 14 14 activating 2020-09-12T14:05:03.479544+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.985 INFO:tasks.workunit.client.0.smithi027.stdout:1.d 3 0 0 0 0 1509 0 0 12 12 activating 2020-09-12T14:05:03.480020+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.985 INFO:tasks.workunit.client.0.smithi027.stdout:1.e 3 1 1 0 0 1156 0 0 14 14 activating 2020-09-12T14:05:03.479738+0000 47'101 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.985 INFO:tasks.workunit.client.0.smithi027.stdout:1.f 3 0 0 0 0 1509 0 0 16 16 activating 2020-09-12T14:05:03.482648+0000 31'100 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:04.985 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:05:04.985 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:05:04.985 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:05:04.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:773: TEST_divergent_3: rados -p test put foo /etc/fstab 2020-09-12T14:05:16.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:777: TEST_divergent_3: echo 'killing divergent 1' 2020-09-12T14:05:16.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:778: TEST_divergent_3: ceph pg dump pgs 2020-09-12T14:05:16.065 INFO:tasks.workunit.client.0.smithi027.stdout:killing divergent 1 2020-09-12T14:05:16.458 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:05:16.459 INFO:tasks.workunit.client.0.smithi027.stdout:1.1f 3 0 0 0 0 1509 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.819956+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.459 INFO:tasks.workunit.client.0.smithi027.stdout:1.1e 3 0 0 0 0 1509 0 0 10 10 active+clean+wait 2020-09-12T14:05:03.817815+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.459 INFO:tasks.workunit.client.0.smithi027.stdout:1.1d 3 0 0 0 0 1509 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.820225+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.459 INFO:tasks.workunit.client.0.smithi027.stdout:1.1c 3 0 0 0 0 1509 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.817525+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.459 INFO:tasks.workunit.client.0.smithi027.stdout:1.1b 3 1 1 0 0 1509 0 0 10 10 active+recovery_wait+degraded+wait 2020-09-12T14:05:03.818764+0000 31'100 52:132 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.459 INFO:tasks.workunit.client.0.smithi027.stdout:1.1a 3 0 0 0 0 1509 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.692308+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.460 INFO:tasks.workunit.client.0.smithi027.stdout:1.19 3 0 0 0 0 1509 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.693556+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.460 INFO:tasks.workunit.client.0.smithi027.stdout:1.18 3 0 0 0 0 1509 0 0 10 10 active+clean+wait 2020-09-12T14:05:03.819902+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.460 INFO:tasks.workunit.client.0.smithi027.stdout:1.17 3 0 0 0 0 1509 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.816919+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.460 INFO:tasks.workunit.client.0.smithi027.stdout:1.16 3 0 0 0 0 1509 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.818444+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.460 INFO:tasks.workunit.client.0.smithi027.stdout:1.15 3 0 0 0 0 1509 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.694233+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.460 INFO:tasks.workunit.client.0.smithi027.stdout:1.14 3 0 0 0 0 1509 0 0 10 10 active+clean+wait 2020-09-12T14:05:03.694190+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.460 INFO:tasks.workunit.client.0.smithi027.stdout:1.13 3 0 0 0 0 1509 0 0 10 10 active+clean+wait 2020-09-12T14:05:03.819609+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.461 INFO:tasks.workunit.client.0.smithi027.stdout:1.12 3 0 0 0 0 1509 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.695064+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.461 INFO:tasks.workunit.client.0.smithi027.stdout:1.11 3 1 1 0 0 1509 0 0 16 16 active+recovery_wait+degraded+wait 2020-09-12T14:05:03.817996+0000 31'100 52:132 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.461 INFO:tasks.workunit.client.0.smithi027.stdout:1.10 3 0 0 0 0 1509 0 0 10 10 active+clean+wait 2020-09-12T14:05:03.819225+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.461 INFO:tasks.workunit.client.0.smithi027.stdout:1.5 3 0 0 0 0 1510 0 0 10 10 active+clean+wait 2020-09-12T14:05:03.694568+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.461 INFO:tasks.workunit.client.0.smithi027.stdout:1.4 4 0 0 0 0 1510 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.693644+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.461 INFO:tasks.workunit.client.0.smithi027.stdout:1.3 4 0 0 0 0 1510 0 0 10 10 active+clean+wait 2020-09-12T14:05:03.694241+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.461 INFO:tasks.workunit.client.0.smithi027.stdout:1.2 4 0 0 0 0 1510 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.694374+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.462 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 3 0 0 0 0 1509 0 0 10 10 active+clean+wait 2020-09-12T14:05:03.818214+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.462 INFO:tasks.workunit.client.0.smithi027.stdout:1.1 4 0 0 0 0 1510 0 0 15 15 active+clean+wait 2020-09-12T14:05:03.819475+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.462 INFO:tasks.workunit.client.0.smithi027.stdout:1.6 3 0 0 0 0 1510 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.819649+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.462 INFO:tasks.workunit.client.0.smithi027.stdout:1.7 3 0 0 0 0 1510 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.693560+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.462 INFO:tasks.workunit.client.0.smithi027.stdout:1.8 3 0 0 0 0 1510 0 0 10 10 active+clean+wait 2020-09-12T14:05:03.693979+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.462 INFO:tasks.workunit.client.0.smithi027.stdout:1.9 3 1 1 0 0 1510 0 0 11 11 active+recovering+degraded+wait 2020-09-12T14:05:03.695518+0000 31'100 52:133 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.462 INFO:tasks.workunit.client.0.smithi027.stdout:1.a 3 0 0 0 0 1510 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.818470+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.463 INFO:tasks.workunit.client.0.smithi027.stdout:1.b 3 0 0 0 0 1510 0 0 10 10 active+clean+wait 2020-09-12T14:05:03.692618+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.463 INFO:tasks.workunit.client.0.smithi027.stdout:1.c 3 0 0 0 0 1510 0 0 14 14 active+clean+wait 2020-09-12T14:05:03.694114+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.463 INFO:tasks.workunit.client.0.smithi027.stdout:1.d 3 0 0 0 0 1509 0 0 12 12 active+clean+wait 2020-09-12T14:05:03.694487+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.463 INFO:tasks.workunit.client.0.smithi027.stdout:1.e 3 1 1 0 0 1156 0 0 14 14 active+recovery_wait+degraded+wait 2020-09-12T14:05:03.693293+0000 47'101 52:132 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.463 INFO:tasks.workunit.client.0.smithi027.stdout:1.f 3 0 0 0 0 1509 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.695010+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:16.463 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:05:16.463 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:05:16.464 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:05:16.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:779: TEST_divergent_3: kill_daemons td/divergent-priors KILL osd.1 2020-09-12T14:05:16.470 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:05:16.470 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:05:16.470 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:05:16.470 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:05:16.470 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:05:16.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:05:16.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:782: TEST_divergent_3: expfile=td/divergent-priors/exp.20815.out 2020-09-12T14:05:16.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:783: TEST_divergent_3: _objectstore_tool_nodown td/divergent-priors 1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.20815.out 2020-09-12T14:05:16.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1195: _objectstore_tool_nodown: local dir=td/divergent-priors 2020-09-12T14:05:16.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: _objectstore_tool_nodown: shift 2020-09-12T14:05:16.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: _objectstore_tool_nodown: local id=1 2020-09-12T14:05:16.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1198: _objectstore_tool_nodown: shift 2020-09-12T14:05:16.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2020-09-12T14:05:16.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1202: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.20815.out 2020-09-12T14:05:17.373 INFO:tasks.workunit.client.0.smithi027.stderr:Exporting 1.0 info 1.0( v 31'100 (31'90,31'100] local-lis/les=51/52 n=3 ec=23/23 lis/c=51/51 les/c/f=52/52/0 sis=51 pruub=12.143260902s) 2020-09-12T14:05:17.388 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:023dd3d2:::existing_12:head# 2020-09-12T14:05:17.406 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:04c40a0c:::existing_93:head# 2020-09-12T14:05:17.424 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:064ff5fb:::existing_36:head# 2020-09-12T14:05:17.425 INFO:tasks.workunit.client.0.smithi027.stderr:Export successful 2020-09-12T14:05:17.425 INFO:tasks.workunit.client.0.smithi027.stdout: marking collection for removal 2020-09-12T14:05:17.426 INFO:tasks.workunit.client.0.smithi027.stdout:setting '_remove' omap key 2020-09-12T14:05:17.426 INFO:tasks.workunit.client.0.smithi027.stdout:finish_remove_pgs 1.0_head removing 1.0 2020-09-12T14:05:17.556 INFO:tasks.workunit.client.0.smithi027.stderr:Remove successful 2020-09-12T14:05:17.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:784: TEST_divergent_3: _objectstore_tool_nodown td/divergent-priors 1 --op import --file td/divergent-priors/exp.20815.out 2020-09-12T14:05:17.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1195: _objectstore_tool_nodown: local dir=td/divergent-priors 2020-09-12T14:05:17.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: _objectstore_tool_nodown: shift 2020-09-12T14:05:17.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: _objectstore_tool_nodown: local id=1 2020-09-12T14:05:17.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1198: _objectstore_tool_nodown: shift 2020-09-12T14:05:17.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2020-09-12T14:05:17.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1202: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op import --file td/divergent-priors/exp.20815.out 2020-09-12T14:05:18.589 INFO:tasks.workunit.client.0.smithi027.stdout:get_pg_num_history pg_num_history pg_num_history(e52 pg_nums {1={23=1,33=32}} deleted_pools ) 2020-09-12T14:05:18.589 INFO:tasks.workunit.client.0.smithi027.stdout:Importing pgid 1.0 2020-09-12T14:05:18.589 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:023dd3d2:::existing_12:head# 2020-09-12T14:05:18.590 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:05:18.691 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:04c40a0c:::existing_93:head# 2020-09-12T14:05:18.691 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:05:18.742 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:064ff5fb:::existing_36:head# 2020-09-12T14:05:18.742 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:05:18.830 INFO:tasks.workunit.client.0.smithi027.stdout:write_pg epoch 52 info 1.0( v 31'100 (31'90,31'100] local-lis/les=51/52 n=3 ec=23/23 lis/c=51/51 les/c/f=52/52/0 sis=51 pruub=12.143260902s) 2020-09-12T14:05:18.900 INFO:tasks.workunit.client.0.smithi027.stdout:Import successful 2020-09-12T14:05:19.422 INFO:tasks.workunit.client.0.smithi027.stdout:reviving divergent 1 2020-09-12T14:05:19.422 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:786: TEST_divergent_3: echo 'reviving divergent 1' 2020-09-12T14:05:19.423 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:787: TEST_divergent_3: ceph pg dump pgs 2020-09-12T14:05:19.818 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:05:19.818 INFO:tasks.workunit.client.0.smithi027.stdout:1.1f 3 0 0 0 0 1509 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.819956+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.818 INFO:tasks.workunit.client.0.smithi027.stdout:1.1e 3 0 0 0 0 1509 0 0 10 10 active+clean+wait 2020-09-12T14:05:03.817815+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.818 INFO:tasks.workunit.client.0.smithi027.stdout:1.1d 3 0 0 0 0 1509 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.820225+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.819 INFO:tasks.workunit.client.0.smithi027.stdout:1.1c 3 0 0 0 0 1509 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.817525+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.819 INFO:tasks.workunit.client.0.smithi027.stdout:1.1b 3 1 1 0 0 1509 0 0 10 10 active+recovery_wait+degraded+wait 2020-09-12T14:05:03.818764+0000 31'100 52:132 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.819 INFO:tasks.workunit.client.0.smithi027.stdout:1.1a 3 0 0 0 0 1509 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.692308+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.819 INFO:tasks.workunit.client.0.smithi027.stdout:1.19 3 0 0 0 0 1509 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.693556+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.819 INFO:tasks.workunit.client.0.smithi027.stdout:1.18 3 0 0 0 0 1509 0 0 10 10 active+clean+wait 2020-09-12T14:05:03.819902+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.819 INFO:tasks.workunit.client.0.smithi027.stdout:1.17 3 0 0 0 0 1509 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.816919+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.819 INFO:tasks.workunit.client.0.smithi027.stdout:1.16 3 0 0 0 0 1509 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.818444+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.819 INFO:tasks.workunit.client.0.smithi027.stdout:1.15 3 0 0 0 0 1509 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.694233+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.820 INFO:tasks.workunit.client.0.smithi027.stdout:1.14 3 0 0 0 0 1509 0 0 10 10 active+clean+wait 2020-09-12T14:05:03.694190+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.820 INFO:tasks.workunit.client.0.smithi027.stdout:1.13 3 0 0 0 0 1509 0 0 10 10 active+clean+wait 2020-09-12T14:05:03.819609+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.820 INFO:tasks.workunit.client.0.smithi027.stdout:1.12 3 0 0 0 0 1509 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.695064+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.820 INFO:tasks.workunit.client.0.smithi027.stdout:1.11 3 1 1 0 0 1509 0 0 16 16 active+recovery_wait+degraded+wait 2020-09-12T14:05:03.817996+0000 31'100 52:132 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.820 INFO:tasks.workunit.client.0.smithi027.stdout:1.10 3 0 0 0 0 1509 0 0 10 10 active+clean+wait 2020-09-12T14:05:03.819225+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.820 INFO:tasks.workunit.client.0.smithi027.stdout:1.5 3 0 0 0 0 1510 0 0 10 10 active+clean+wait 2020-09-12T14:05:03.694568+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.820 INFO:tasks.workunit.client.0.smithi027.stdout:1.4 4 0 0 0 0 1510 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.693644+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.821 INFO:tasks.workunit.client.0.smithi027.stdout:1.3 4 0 0 0 0 1510 0 0 10 10 active+clean+wait 2020-09-12T14:05:03.694241+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.821 INFO:tasks.workunit.client.0.smithi027.stdout:1.2 4 0 0 0 0 1510 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.694374+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.821 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 3 0 0 0 0 1509 0 0 10 10 active+clean+wait 2020-09-12T14:05:03.818214+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.821 INFO:tasks.workunit.client.0.smithi027.stdout:1.1 4 0 0 0 0 1510 0 0 15 15 active+clean+wait 2020-09-12T14:05:03.819475+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.821 INFO:tasks.workunit.client.0.smithi027.stdout:1.6 3 0 0 0 0 1510 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.819649+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.821 INFO:tasks.workunit.client.0.smithi027.stdout:1.7 3 0 0 0 0 1510 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.693560+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.821 INFO:tasks.workunit.client.0.smithi027.stdout:1.8 3 0 0 0 0 1510 0 0 10 10 active+clean+wait 2020-09-12T14:05:03.693979+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.822 INFO:tasks.workunit.client.0.smithi027.stdout:1.9 3 1 1 0 0 1510 0 0 11 11 active+recovering+degraded+wait 2020-09-12T14:05:03.695518+0000 31'100 52:133 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.822 INFO:tasks.workunit.client.0.smithi027.stdout:1.a 3 0 0 0 0 1510 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.818470+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.822 INFO:tasks.workunit.client.0.smithi027.stdout:1.b 3 0 0 0 0 1510 0 0 10 10 active+clean+wait 2020-09-12T14:05:03.692618+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.822 INFO:tasks.workunit.client.0.smithi027.stdout:1.c 3 0 0 0 0 1510 0 0 14 14 active+clean+wait 2020-09-12T14:05:03.694114+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.822 INFO:tasks.workunit.client.0.smithi027.stdout:1.d 3 0 0 0 0 1509 0 0 12 12 active+clean+wait 2020-09-12T14:05:03.694487+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.822 INFO:tasks.workunit.client.0.smithi027.stdout:1.e 3 1 1 0 0 1156 0 0 14 14 active+recovery_wait+degraded+wait 2020-09-12T14:05:03.693293+0000 47'101 52:132 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.822 INFO:tasks.workunit.client.0.smithi027.stdout:1.f 3 0 0 0 0 1509 0 0 16 16 active+clean+wait 2020-09-12T14:05:03.695010+0000 31'100 52:134 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:19.822 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:05:19.823 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:05:19.823 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:05:19.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:788: TEST_divergent_3: activate_osd td/divergent-priors 1 2020-09-12T14:05:19.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/divergent-priors 2020-09-12T14:05:19.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T14:05:19.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=1 2020-09-12T14:05:19.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T14:05:19.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/divergent-priors/1 2020-09-12T14:05:19.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T14:05:19.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:05:19.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:05:19.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:05:19.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2020-09-12T14:05:19.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2020-09-12T14:05:19.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T14:05:19.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T14:05:19.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T14:05:19.832 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T14:05:19.832 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:05:19.832 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:05:19.832 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:05:19.833 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:05:19.833 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:05:19.833 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:05:19.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:05:19.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:05:19.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T14:05:19.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T14:05:19.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:05:19.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:05:19.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:05:19.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T14:05:19.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T14:05:19.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/divergent-priors/1 2020-09-12T14:05:19.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.1 2020-09-12T14:05:19.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 1 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:05:19.836 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/divergent-priors/1/whoami 2020-09-12T14:05:19.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 1 = 1 ']' 2020-09-12T14:05:19.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T14:05:19.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T14:05:19.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T14:05:19.837 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T14:05:19.860 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:19.859+0000 7fe3fdb92ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:05:19.862 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:19.860+0000 7fe3fdb92ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:05:19.863 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:19.862+0000 7fe3fdb92ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:05:20.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: wait_for_osd up 1 2020-09-12T14:05:20.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:05:20.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:05:20.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:05:20.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:05:20.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:05:20.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:05:20.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:05:20.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:05:20.243 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:05:20.382 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:20.380+0000 7fe3fdb92ec0 -1 Falling back to public interface 2020-09-12T14:05:20.691 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 51 up_thru 51 down_at 36 last_clean_interval [10,34) [v2:127.0.0.1:6818/52023,v1:127.0.0.1:6819/52023] [v2:127.0.0.1:6820/52023,v1:127.0.0.1:6821/52023] exists,up 06975646-fe2b-4ca5-82e0-a2d0e01e06a8 2020-09-12T14:05:20.692 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:05:20.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:05:20.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:05:20.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:05:20.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:789: TEST_divergent_3: wait_for_osd up 1 2020-09-12T14:05:20.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:05:20.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:05:20.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:05:20.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:05:20.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:05:20.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:05:20.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:05:20.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:05:21.102 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 51 up_thru 51 down_at 36 last_clean_interval [10,34) [v2:127.0.0.1:6818/52023,v1:127.0.0.1:6819/52023] [v2:127.0.0.1:6820/52023,v1:127.0.0.1:6821/52023] exists,up 06975646-fe2b-4ca5-82e0-a2d0e01e06a8 2020-09-12T14:05:21.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:05:21.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:05:21.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:05:21.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:791: TEST_divergent_3: sleep 20 2020-09-12T14:05:22.167 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:22.165+0000 7fe3fdb92ec0 -1 osd.1 52 log_to_monitors {default=true} 2020-09-12T14:05:41.104 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:792: TEST_divergent_3: get_asok_path osd.1 2020-09-12T14:05:41.105 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2020-09-12T14:05:41.105 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2020-09-12T14:05:41.105 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:05:41.105 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:05:41.106 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:05:41.106 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.1.asok 2020-09-12T14:05:41.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:792: TEST_divergent_3: CEPH_ARGS= 2020-09-12T14:05:41.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:792: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.1.asok dump_ops_in_flight 2020-09-12T14:05:41.219 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:05:41.220 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [], 2020-09-12T14:05:41.220 INFO:tasks.workunit.client.0.smithi027.stdout: "num_ops": 0 2020-09-12T14:05:41.221 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:05:41.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:794: TEST_divergent_3: echo 'allowing recovery' 2020-09-12T14:05:41.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:795: TEST_divergent_3: ceph pg dump pgs 2020-09-12T14:05:41.229 INFO:tasks.workunit.client.0.smithi027.stdout:allowing recovery 2020-09-12T14:05:41.619 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:05:41.619 INFO:tasks.workunit.client.0.smithi027.stdout:1.1f 3 0 0 0 0 1509 0 0 16 16 active+clean 2020-09-12T14:05:33.629198+0000 31'100 62:157 [1,2,0] 1 [1,2,0] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.619 INFO:tasks.workunit.client.0.smithi027.stdout:1.1e 3 0 0 0 0 1509 0 0 16 16 active+clean 2020-09-12T14:05:31.780112+0000 31'100 61:144 [0,1,2] 0 [0,1,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.620 INFO:tasks.workunit.client.0.smithi027.stdout:1.1d 3 0 0 0 0 1509 0 0 16 16 active+clean 2020-09-12T14:05:29.890422+0000 31'100 63:128 [2,0,1] 2 [2,0,1] 2 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.620 INFO:tasks.workunit.client.0.smithi027.stdout:1.1c 3 0 0 0 0 1509 0 0 16 16 active+clean 2020-09-12T14:05:23.717147+0000 31'100 62:152 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.620 INFO:tasks.workunit.client.0.smithi027.stdout:1.1b 3 0 0 0 0 1509 0 0 10 10 active+clean 2020-09-12T14:05:33.628123+0000 31'100 62:161 [1,2,0] 1 [1,2,0] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.620 INFO:tasks.workunit.client.0.smithi027.stdout:1.1a 3 0 0 0 0 1509 0 0 16 16 active+clean 2020-09-12T14:05:31.740185+0000 31'100 65:148 [0,1,2] 0 [0,1,2] 0 31'100 2020-09-12T14:05:30.141621+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.621 INFO:tasks.workunit.client.0.smithi027.stdout:1.19 3 0 0 0 0 1509 0 0 16 16 active+clean 2020-09-12T14:05:29.863563+0000 31'100 63:128 [2,0,1] 2 [2,0,1] 2 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.621 INFO:tasks.workunit.client.0.smithi027.stdout:1.18 3 0 0 0 0 1509 0 0 10 10 active+clean 2020-09-12T14:05:23.719530+0000 31'100 62:153 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.621 INFO:tasks.workunit.client.0.smithi027.stdout:1.17 3 0 0 0 0 1509 0 0 16 16 active+clean 2020-09-12T14:05:33.629407+0000 31'100 62:157 [1,2,0] 1 [1,2,0] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.621 INFO:tasks.workunit.client.0.smithi027.stdout:1.16 3 0 0 0 0 1509 0 0 16 16 active+clean 2020-09-12T14:05:31.780614+0000 31'100 61:144 [0,1,2] 0 [0,1,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.622 INFO:tasks.workunit.client.0.smithi027.stdout:1.15 3 0 0 0 0 1449 0 0 16 16 active+clean 2020-09-12T14:05:33.795569+0000 31'100 63:131 [2,0,1] 2 [2,0,1] 2 31'100 2020-09-12T14:05:33.795506+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.622 INFO:tasks.workunit.client.0.smithi027.stdout:1.14 3 0 0 0 0 1509 0 0 10 10 active+clean 2020-09-12T14:05:23.719053+0000 31'100 62:153 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.622 INFO:tasks.workunit.client.0.smithi027.stdout:1.13 3 0 0 0 0 1509 0 0 10 10 active+clean 2020-09-12T14:05:33.628031+0000 31'100 62:159 [1,2,0] 1 [1,2,0] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.622 INFO:tasks.workunit.client.0.smithi027.stdout:1.12 3 0 0 0 0 1509 0 0 16 16 active+clean 2020-09-12T14:05:31.779331+0000 31'100 65:148 [0,1,2] 0 [0,1,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.623 INFO:tasks.workunit.client.0.smithi027.stdout:1.11 3 0 0 0 0 1509 0 0 16 16 active+clean 2020-09-12T14:05:29.888837+0000 31'100 63:128 [2,0,1] 2 [2,0,1] 2 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.623 INFO:tasks.workunit.client.0.smithi027.stdout:1.10 3 0 0 0 0 1509 0 0 10 10 active+clean 2020-09-12T14:05:23.719666+0000 31'100 62:153 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.623 INFO:tasks.workunit.client.0.smithi027.stdout:1.5 6 0 0 0 0 2898 0 0 16 16 active+clean 2020-09-12T14:05:35.770482+0000 31'100 63:131 [2,0,1] 2 [2,0,1] 2 31'100 2020-09-12T14:05:35.770433+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.624 INFO:tasks.workunit.client.0.smithi027.stdout:1.4 4 0 0 0 0 1510 0 0 16 16 active+clean 2020-09-12T14:05:23.718276+0000 31'100 62:152 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.624 INFO:tasks.workunit.client.0.smithi027.stdout:1.3 4 0 0 0 0 1510 0 0 10 10 active+clean 2020-09-12T14:05:33.628932+0000 31'100 62:159 [1,2,0] 1 [1,2,0] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.624 INFO:tasks.workunit.client.0.smithi027.stdout:1.2 1 0 0 0 0 483 0 0 16 16 active+clean 2020-09-12T14:05:35.095454+0000 31'100 65:151 [0,1,2] 0 [0,1,2] 0 31'100 2020-09-12T14:05:35.095401+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.624 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 3 0 0 0 0 1509 0 0 10 10 active+clean 2020-09-12T14:05:23.720310+0000 31'100 62:153 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.625 INFO:tasks.workunit.client.0.smithi027.stdout:1.1 4 0 0 0 0 1510 0 0 16 16 active+clean 2020-09-12T14:05:29.889969+0000 31'100 63:128 [2,0,1] 2 [2,0,1] 2 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.625 INFO:tasks.workunit.client.0.smithi027.stdout:1.6 4 0 0 0 0 1993 0 0 17 17 active+clean 2020-09-12T14:05:31.780843+0000 52'101 61:150 [0,1,2] 0 [0,1,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.625 INFO:tasks.workunit.client.0.smithi027.stdout:1.7 2 0 0 0 0 966 0 0 16 16 active+clean 2020-09-12T14:05:33.594175+0000 31'100 62:160 [1,2,0] 1 [1,2,0] 1 31'100 2020-09-12T14:05:28.172688+0000 31'100 2020-09-12T14:05:28.172688+0000 0 2020-09-12T14:05:41.626 INFO:tasks.workunit.client.0.smithi027.stdout:1.8 3 0 0 0 0 1510 0 0 10 10 active+clean 2020-09-12T14:05:23.714046+0000 31'100 62:153 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.626 INFO:tasks.workunit.client.0.smithi027.stdout:1.9 5 0 0 0 0 2415 0 0 16 16 active+clean 2020-09-12T14:05:30.685635+0000 31'100 63:131 [2,0,1] 2 [2,0,1] 2 31'100 2020-09-12T14:05:30.685573+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.626 INFO:tasks.workunit.client.0.smithi027.stdout:1.a 3 0 0 0 0 1510 0 0 16 16 active+clean 2020-09-12T14:05:31.780383+0000 31'100 65:148 [0,1,2] 0 [0,1,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.626 INFO:tasks.workunit.client.0.smithi027.stdout:1.b 3 0 0 0 0 1510 0 0 10 10 active+clean 2020-09-12T14:05:33.628830+0000 31'100 62:159 [1,2,0] 1 [1,2,0] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.626 INFO:tasks.workunit.client.0.smithi027.stdout:1.c 3 0 0 0 0 1510 0 0 14 14 active+clean 2020-09-12T14:05:23.716707+0000 31'100 62:153 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.626 INFO:tasks.workunit.client.0.smithi027.stdout:1.d 3 0 0 0 0 1449 0 0 16 16 active+clean 2020-09-12T14:05:36.740406+0000 31'100 63:131 [2,0,1] 2 [2,0,1] 2 31'100 2020-09-12T14:05:36.740358+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.626 INFO:tasks.workunit.client.0.smithi027.stdout:1.e 3 0 0 0 0 1156 0 0 17 17 active+clean 2020-09-12T14:05:31.779463+0000 47'101 61:145 [0,1,2] 0 [0,1,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.627 INFO:tasks.workunit.client.0.smithi027.stdout:1.f 3 0 0 0 0 1509 0 0 16 16 active+clean 2020-09-12T14:05:33.627867+0000 31'100 62:157 [1,2,0] 1 [1,2,0] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:41.627 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:05:41.627 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:05:41.627 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:05:41.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:797: TEST_divergent_3: for i in $osds 2020-09-12T14:05:41.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:799: TEST_divergent_3: ceph tell osd.0 debug kick_recovery_wq 0 2020-09-12T14:05:41.767 INFO:tasks.workunit.client.0.smithi027.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2020-09-12T14:05:41.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:797: TEST_divergent_3: for i in $osds 2020-09-12T14:05:41.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:799: TEST_divergent_3: ceph tell osd.1 debug kick_recovery_wq 0 2020-09-12T14:05:41.914 INFO:tasks.workunit.client.0.smithi027.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2020-09-12T14:05:41.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:797: TEST_divergent_3: for i in $osds 2020-09-12T14:05:41.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:799: TEST_divergent_3: ceph tell osd.2 debug kick_recovery_wq 0 2020-09-12T14:05:42.095 INFO:tasks.workunit.client.0.smithi027.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2020-09-12T14:05:42.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:802: TEST_divergent_3: echo 'reading divergent objects' 2020-09-12T14:05:42.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:803: TEST_divergent_3: ceph pg dump pgs 2020-09-12T14:05:42.104 INFO:tasks.workunit.client.0.smithi027.stdout:reading divergent objects 2020-09-12T14:05:42.491 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:05:42.491 INFO:tasks.workunit.client.0.smithi027.stdout:1.1f 3 0 0 0 0 1509 0 0 16 16 active+clean 2020-09-12T14:05:33.629198+0000 31'100 62:157 [1,2,0] 1 [1,2,0] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.492 INFO:tasks.workunit.client.0.smithi027.stdout:1.1e 3 0 0 0 0 1509 0 0 16 16 active+clean 2020-09-12T14:05:31.780112+0000 31'100 61:144 [0,1,2] 0 [0,1,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.492 INFO:tasks.workunit.client.0.smithi027.stdout:1.1d 3 0 0 0 0 1509 0 0 16 16 active+clean 2020-09-12T14:05:29.890422+0000 31'100 63:128 [2,0,1] 2 [2,0,1] 2 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.492 INFO:tasks.workunit.client.0.smithi027.stdout:1.1c 3 0 0 0 0 1509 0 0 16 16 active+clean 2020-09-12T14:05:23.717147+0000 31'100 62:152 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.492 INFO:tasks.workunit.client.0.smithi027.stdout:1.1b 3 0 0 0 0 1509 0 0 10 10 active+clean 2020-09-12T14:05:33.628123+0000 31'100 62:161 [1,2,0] 1 [1,2,0] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.492 INFO:tasks.workunit.client.0.smithi027.stdout:1.1a 3 0 0 0 0 1509 0 0 16 16 active+clean 2020-09-12T14:05:31.740185+0000 31'100 65:148 [0,1,2] 0 [0,1,2] 0 31'100 2020-09-12T14:05:30.141621+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.492 INFO:tasks.workunit.client.0.smithi027.stdout:1.19 3 0 0 0 0 1509 0 0 16 16 active+clean 2020-09-12T14:05:29.863563+0000 31'100 63:128 [2,0,1] 2 [2,0,1] 2 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.493 INFO:tasks.workunit.client.0.smithi027.stdout:1.18 3 0 0 0 0 1509 0 0 10 10 active+clean 2020-09-12T14:05:23.719530+0000 31'100 62:153 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.493 INFO:tasks.workunit.client.0.smithi027.stdout:1.17 3 0 0 0 0 1509 0 0 16 16 active+clean 2020-09-12T14:05:33.629407+0000 31'100 62:157 [1,2,0] 1 [1,2,0] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.493 INFO:tasks.workunit.client.0.smithi027.stdout:1.16 3 0 0 0 0 1509 0 0 16 16 active+clean 2020-09-12T14:05:31.780614+0000 31'100 61:144 [0,1,2] 0 [0,1,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.493 INFO:tasks.workunit.client.0.smithi027.stdout:1.15 3 0 0 0 0 1449 0 0 16 16 active+clean 2020-09-12T14:05:33.795569+0000 31'100 63:131 [2,0,1] 2 [2,0,1] 2 31'100 2020-09-12T14:05:33.795506+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.493 INFO:tasks.workunit.client.0.smithi027.stdout:1.14 3 0 0 0 0 1509 0 0 10 10 active+clean 2020-09-12T14:05:23.719053+0000 31'100 62:153 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.493 INFO:tasks.workunit.client.0.smithi027.stdout:1.13 3 0 0 0 0 1509 0 0 10 10 active+clean 2020-09-12T14:05:33.628031+0000 31'100 62:159 [1,2,0] 1 [1,2,0] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.493 INFO:tasks.workunit.client.0.smithi027.stdout:1.12 3 0 0 0 0 1509 0 0 16 16 active+clean 2020-09-12T14:05:31.779331+0000 31'100 65:148 [0,1,2] 0 [0,1,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.494 INFO:tasks.workunit.client.0.smithi027.stdout:1.11 3 0 0 0 0 1509 0 0 16 16 active+clean 2020-09-12T14:05:29.888837+0000 31'100 63:128 [2,0,1] 2 [2,0,1] 2 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.494 INFO:tasks.workunit.client.0.smithi027.stdout:1.10 3 0 0 0 0 1509 0 0 10 10 active+clean 2020-09-12T14:05:23.719666+0000 31'100 62:153 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.494 INFO:tasks.workunit.client.0.smithi027.stdout:1.5 6 0 0 0 0 2898 0 0 16 16 active+clean 2020-09-12T14:05:35.770482+0000 31'100 63:131 [2,0,1] 2 [2,0,1] 2 31'100 2020-09-12T14:05:35.770433+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.494 INFO:tasks.workunit.client.0.smithi027.stdout:1.4 4 0 0 0 0 1510 0 0 16 16 active+clean 2020-09-12T14:05:23.718276+0000 31'100 62:152 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.494 INFO:tasks.workunit.client.0.smithi027.stdout:1.3 4 0 0 0 0 1510 0 0 10 10 active+clean 2020-09-12T14:05:33.628932+0000 31'100 62:159 [1,2,0] 1 [1,2,0] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.494 INFO:tasks.workunit.client.0.smithi027.stdout:1.2 1 0 0 0 0 483 0 0 16 16 active+clean 2020-09-12T14:05:35.095454+0000 31'100 65:151 [0,1,2] 0 [0,1,2] 0 31'100 2020-09-12T14:05:35.095401+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.494 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 3 0 0 0 0 1509 0 0 10 10 active+clean 2020-09-12T14:05:23.720310+0000 31'100 62:153 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.495 INFO:tasks.workunit.client.0.smithi027.stdout:1.1 4 0 0 0 0 1510 0 0 16 16 active+clean 2020-09-12T14:05:29.889969+0000 31'100 63:128 [2,0,1] 2 [2,0,1] 2 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.495 INFO:tasks.workunit.client.0.smithi027.stdout:1.6 4 0 0 0 0 1993 0 0 17 17 active+clean 2020-09-12T14:05:31.780843+0000 52'101 61:150 [0,1,2] 0 [0,1,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.495 INFO:tasks.workunit.client.0.smithi027.stdout:1.7 2 0 0 0 0 966 0 0 16 16 active+clean 2020-09-12T14:05:33.594175+0000 31'100 62:160 [1,2,0] 1 [1,2,0] 1 31'100 2020-09-12T14:05:28.172688+0000 31'100 2020-09-12T14:05:28.172688+0000 0 2020-09-12T14:05:42.495 INFO:tasks.workunit.client.0.smithi027.stdout:1.8 3 0 0 0 0 1510 0 0 10 10 active+clean 2020-09-12T14:05:23.714046+0000 31'100 62:153 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.495 INFO:tasks.workunit.client.0.smithi027.stdout:1.9 5 0 0 0 0 2415 0 0 16 16 active+clean 2020-09-12T14:05:30.685635+0000 31'100 63:131 [2,0,1] 2 [2,0,1] 2 31'100 2020-09-12T14:05:30.685573+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.495 INFO:tasks.workunit.client.0.smithi027.stdout:1.a 3 0 0 0 0 1510 0 0 16 16 active+clean 2020-09-12T14:05:31.780383+0000 31'100 65:148 [0,1,2] 0 [0,1,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.495 INFO:tasks.workunit.client.0.smithi027.stdout:1.b 3 0 0 0 0 1510 0 0 10 10 active+clean 2020-09-12T14:05:33.628830+0000 31'100 62:159 [1,2,0] 1 [1,2,0] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.496 INFO:tasks.workunit.client.0.smithi027.stdout:1.c 3 0 0 0 0 1510 0 0 14 14 active+clean 2020-09-12T14:05:23.716707+0000 31'100 62:153 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.496 INFO:tasks.workunit.client.0.smithi027.stdout:1.d 3 0 0 0 0 1449 0 0 16 16 active+clean 2020-09-12T14:05:36.740406+0000 31'100 63:131 [2,0,1] 2 [2,0,1] 2 31'100 2020-09-12T14:05:36.740358+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.496 INFO:tasks.workunit.client.0.smithi027.stdout:1.e 3 0 0 0 0 1156 0 0 17 17 active+clean 2020-09-12T14:05:31.779463+0000 47'101 61:145 [0,1,2] 0 [0,1,2] 0 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.496 INFO:tasks.workunit.client.0.smithi027.stdout:1.f 3 0 0 0 0 1509 0 0 16 16 active+clean 2020-09-12T14:05:33.627867+0000 31'100 62:157 [1,2,0] 1 [1,2,0] 1 0'0 2020-09-12T14:02:45.582986+0000 0'0 2020-09-12T14:02:45.582986+0000 0 2020-09-12T14:05:42.496 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:05:42.496 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:05:42.497 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:05:42.502 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:804: TEST_divergent_3: expr 2 + 2 2020-09-12T14:05:42.504 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:804: TEST_divergent_3: seq 1 4 2020-09-12T14:05:42.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:804: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2020-09-12T14:05:42.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:806: TEST_divergent_3: rados -p test get existing_1 td/divergent-priors/existing 2020-09-12T14:05:42.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:804: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2020-09-12T14:05:42.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:806: TEST_divergent_3: rados -p test get existing_2 td/divergent-priors/existing 2020-09-12T14:05:42.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:804: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2020-09-12T14:05:42.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:806: TEST_divergent_3: rados -p test get existing_3 td/divergent-priors/existing 2020-09-12T14:05:42.616 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:804: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2020-09-12T14:05:42.616 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:806: TEST_divergent_3: rados -p test get existing_4 td/divergent-priors/existing 2020-09-12T14:05:42.647 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:808: TEST_divergent_3: seq 1 2 2020-09-12T14:05:42.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:808: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2020-09-12T14:05:42.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:810: TEST_divergent_3: rados -p test get newobject_1 td/divergent-priors/existing 2020-09-12T14:05:42.678 INFO:tasks.workunit.client.0.smithi027.stderr:error getting test/newobject_1: (2) No such file or directory 2020-09-12T14:05:42.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:808: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2020-09-12T14:05:42.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:810: TEST_divergent_3: rados -p test get newobject_2 td/divergent-priors/existing 2020-09-12T14:05:42.708 INFO:tasks.workunit.client.0.smithi027.stderr:error getting test/newobject_2: (2) No such file or directory 2020-09-12T14:05:42.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:812: TEST_divergent_3: rm -f td/divergent-priors/existing 2020-09-12T14:05:42.710 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:814: TEST_divergent_3: find td/divergent-priors -name '*osd*log' 2020-09-12T14:05:42.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:814: TEST_divergent_3: grep _merge_object_divergent_entries td/divergent-priors/osd.2.log td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log 2020-09-12T14:05:42.724 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.177+0000 7f44bbedc700 20 _merge_object_divergent_entries: merging hoid 1:97db2a0f:::existing_2:head entries: 31'104 (31'2) modify 1:97db2a0f:::existing_2:head by client.4537.0:1 2020-09-12T14:03:19.181873+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0 2020-09-12T14:05:42.725 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.177+0000 7f44bbedc700 20 _merge_object_divergent_entries: keeping 31'104 (31'2) modify 1:97db2a0f:::existing_2:head by client.4537.0:1 2020-09-12T14:03:19.181873+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0 2020-09-12T14:05:42.725 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.177+0000 7f44bbedc700 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2020-09-12T14:05:42.725 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.177+0000 7f44bbedc700 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version: 31'2 first_divergent_update: 31'104 last_divergent_update: 31'104 2020-09-12T14:05:42.726 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.177+0000 7f44bbedc700 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head has no more recent entries in log 2020-09-12T14:05:42.726 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.177+0000 7f44bbedc700 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head must be rolled back or recovered, attempting to rollback 2020-09-12T14:05:42.726 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.177+0000 7f44bbedc700 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head olog_can_rollback_to: 31'106 2020-09-12T14:05:42.726 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.177+0000 7f44bbedc700 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot rollback 31'104 (31'2) modify 1:97db2a0f:::existing_2:head by client.4537.0:1 2020-09-12T14:03:19.181873+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0 2020-09-12T14:05:42.726 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.177+0000 7f44bbedc700 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot roll back, removing and adding to missing 2020-09-12T14:05:42.727 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.177+0000 7f44bbedc700 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version 31'2 <= info.log_tail 31'89 2020-09-12T14:05:42.727 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.260+0000 7f44bbedc700 20 _merge_object_divergent_entries: merging hoid 1:70b3bf12:::existing_4:head entries: 31'106 (31'4) delete 1:70b3bf12:::existing_4:head by client.4543.0:1 2020-09-12T14:03:19.186266+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 0, new_object: 0 2020-09-12T14:05:42.727 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.260+0000 7f44bbedc700 20 _merge_object_divergent_entries: keeping 31'106 (31'4) delete 1:70b3bf12:::existing_4:head by client.4543.0:1 2020-09-12T14:03:19.186266+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 0, new_object: 0 2020-09-12T14:05:42.727 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.260+0000 7f44bbedc700 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2020-09-12T14:05:42.727 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.260+0000 7f44bbedc700 10 _merge_object_divergent_entries: hoid 1:70b3bf12:::existing_4:head prior_version: 31'4 first_divergent_update: 31'106 last_divergent_update: 31'106 2020-09-12T14:05:42.727 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.260+0000 7f44bbedc700 10 _merge_object_divergent_entries: more recent entry found: 47'101 (31'4) modify 1:70b3bf12:::existing_4:head by client.4652.0:1 2020-09-12T14:04:17.050322+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0, already merged 2020-09-12T14:05:42.728 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.260+0000 7f44bbedc700 20 _merge_object_divergent_entries: merging hoid 1:75ac5091:::newobject_2:head entries: 31'101 (0'0) modify 1:75ac5091:::newobject_2:head by client.4530.0:1 2020-09-12T14:03:19.164150+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 2020-09-12T14:05:42.728 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.260+0000 7f44bbedc700 20 _merge_object_divergent_entries: keeping 31'101 (0'0) modify 1:75ac5091:::newobject_2:head by client.4530.0:1 2020-09-12T14:03:19.164150+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 2020-09-12T14:05:42.728 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.260+0000 7f44bbedc700 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2020-09-12T14:05:42.728 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.260+0000 7f44bbedc700 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version: 0'0 first_divergent_update: 31'101 last_divergent_update: 31'101 2020-09-12T14:05:42.728 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.260+0000 7f44bbedc700 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head has no more recent entries in log 2020-09-12T14:05:42.728 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.260+0000 7f44bbedc700 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version or op type indicates creation, deleting 2020-09-12T14:05:42.729 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.260+0000 7f44bcede700 20 _merge_object_divergent_entries: merging hoid 1:8881e6d8:::existing_3:head entries: 31'105 (31'3) delete 1:8881e6d8:::existing_3:head by client.4542.0:1 2020-09-12T14:03:19.184900+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 0, new_object: 0 2020-09-12T14:05:42.729 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.260+0000 7f44bcede700 20 _merge_object_divergent_entries: keeping 31'105 (31'3) delete 1:8881e6d8:::existing_3:head by client.4542.0:1 2020-09-12T14:03:19.184900+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 0, new_object: 0 2020-09-12T14:05:42.729 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.260+0000 7f44bcede700 10 _merge_object_divergent_entries: hoid object_not_in_store: 1 2020-09-12T14:05:42.729 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.260+0000 7f44bcede700 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version: 31'3 first_divergent_update: 31'105 last_divergent_update: 31'105 2020-09-12T14:05:42.729 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.260+0000 7f44bcede700 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head has no more recent entries in log 2020-09-12T14:05:42.729 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.260+0000 7f44bcede700 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head must be rolled back or recovered, attempting to rollback 2020-09-12T14:05:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.260+0000 7f44bcede700 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head olog_can_rollback_to: 31'106 2020-09-12T14:05:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.260+0000 7f44bcede700 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot rollback 31'105 (31'3) delete 1:8881e6d8:::existing_3:head by client.4542.0:1 2020-09-12T14:03:19.184900+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 0, new_object: 0 2020-09-12T14:05:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.260+0000 7f44bcede700 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot roll back, removing and adding to missing 2020-09-12T14:05:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.260+0000 7f44bcede700 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version 31'3 <= info.log_tail 31'84 2020-09-12T14:05:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.261+0000 7f44bcede700 20 _merge_object_divergent_entries: merging hoid 1:d858c035:::existing_1:head entries: 31'103 (31'1) modify 1:d858c035:::existing_1:head by client.4538.0:1 2020-09-12T14:03:19.181426+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0 2020-09-12T14:05:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.261+0000 7f44bcede700 20 _merge_object_divergent_entries: keeping 31'103 (31'1) modify 1:d858c035:::existing_1:head by client.4538.0:1 2020-09-12T14:03:19.181426+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0 2020-09-12T14:05:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.261+0000 7f44bcede700 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2020-09-12T14:05:42.731 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.261+0000 7f44bcede700 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version: 31'1 first_divergent_update: 31'103 last_divergent_update: 31'103 2020-09-12T14:05:42.731 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.261+0000 7f44bcede700 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head has no more recent entries in log 2020-09-12T14:05:42.731 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.261+0000 7f44bcede700 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head must be rolled back or recovered, attempting to rollback 2020-09-12T14:05:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.261+0000 7f44bcede700 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head olog_can_rollback_to: 31'106 2020-09-12T14:05:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.261+0000 7f44bcede700 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot rollback 31'103 (31'1) modify 1:d858c035:::existing_1:head by client.4538.0:1 2020-09-12T14:03:19.181426+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0 2020-09-12T14:05:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.261+0000 7f44bcede700 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot roll back, removing and adding to missing 2020-09-12T14:05:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.261+0000 7f44bcede700 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version 31'1 <= info.log_tail 31'90 2020-09-12T14:05:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.261+0000 7f44bcede700 20 _merge_object_divergent_entries: merging hoid 1:da59f6e7:::newobject_1:head entries: 31'102 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4531.0:1 2020-09-12T14:03:19.165966+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 2020-09-12T14:05:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.261+0000 7f44bcede700 20 _merge_object_divergent_entries: keeping 31'102 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4531.0:1 2020-09-12T14:03:19.165966+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 2020-09-12T14:05:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.261+0000 7f44bcede700 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2020-09-12T14:05:42.733 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.261+0000 7f44bcede700 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version: 0'0 first_divergent_update: 31'102 last_divergent_update: 31'102 2020-09-12T14:05:42.733 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.261+0000 7f44bcede700 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head has no more recent entries in log 2020-09-12T14:05:42.733 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:05:03.261+0000 7f44bcede700 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version or op type indicates creation, deleting 2020-09-12T14:05:42.733 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:816: TEST_divergent_3: find td/divergent-priors -name '*osd*log' 2020-09-12T14:05:42.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:816: TEST_divergent_3: grep -q '_merge_object_divergent_entries: more recent entry found:' td/divergent-priors/osd.2.log td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log 2020-09-12T14:05:42.740 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:822: TEST_divergent_3: find td/divergent-priors -name '*osd*log' 2020-09-12T14:05:42.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:822: TEST_divergent_3: grep -q '_merge_object_divergent_entries.*prior_version or op type indicates creation' td/divergent-priors/osd.2.log td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log 2020-09-12T14:05:42.749 INFO:tasks.workunit.client.0.smithi027.stdout:success 2020-09-12T14:05:42.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:827: TEST_divergent_3: echo success 2020-09-12T14:05:42.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:829: TEST_divergent_3: rm td/divergent-priors/td/divergent-priors/exp.20815.out 2020-09-12T14:05:42.751 INFO:tasks.workunit.client.0.smithi027.stderr:rm: cannot remove 'td/divergent-priors/td/divergent-priors/exp.20815.out': No such file or directory 2020-09-12T14:05:42.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:831: TEST_divergent_3: delete_pool test 2020-09-12T14:05:42.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: delete_pool: local poolname=test 2020-09-12T14:05:42.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2020-09-12T14:05:43.635 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test' does not exist 2020-09-12T14:05:43.644 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:832: TEST_divergent_3: kill_daemons td/divergent-priors 2020-09-12T14:05:43.645 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:05:43.645 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:05:43.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:05:43.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:05:43.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:05:43.953 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:05:43.953 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2020-09-12T14:05:43.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2020-09-12T14:05:43.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:05:43.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2020-09-12T14:05:43.955 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:05:43.955 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:05:43.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:05:43.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:05:43.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:05:43.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:05:43.959 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:05:43.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:05:43.960 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:05:43.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:05:43.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:05:43.961 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:05:43.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:05:43.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:05:43.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:05:43.964 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:05:43.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:05:43.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:05:43.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2020-09-12T14:05:43.982 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:05:43.983 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:05:43.983 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:05:43.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.20815 2020-09-12T14:05:43.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:05:43.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:05:43.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2020-09-12T14:05:43.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2020-09-12T14:05:43.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/divergent-priors 2020-09-12T14:05:43.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/divergent-priors 2020-09-12T14:05:43.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2020-09-12T14:05:43.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:05:43.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2020-09-12T14:05:43.985 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:05:43.986 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:05:43.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:05:43.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:05:43.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:05:43.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:05:43.987 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:05:43.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:05:43.987 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:05:43.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:05:43.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:05:43.988 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:05:43.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:05:43.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:05:43.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:05:43.991 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:05:43.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:05:43.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:05:43.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2020-09-12T14:05:43.993 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:05:43.993 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:05:43.994 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:05:43.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.20815 2020-09-12T14:05:43.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:05:43.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:05:43.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/divergent-priors 2020-09-12T14:05:43.995 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2020-09-12T14:05:43.996 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:05:43.996 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:05:43.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.20815 2020-09-12T14:05:43.997 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2020-09-12T14:05:43.997 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2020-09-12T14:05:43.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2020-09-12T14:05:43.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2020-09-12T14:05:43.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent_ec td/divergent-priors 2020-09-12T14:05:43.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:228: TEST_divergent_ec: local dir=td/divergent-priors 2020-09-12T14:05:43.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:231: TEST_divergent_ec: local dummyfile=/etc/fstab 2020-09-12T14:05:43.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:232: TEST_divergent_ec: local dummyfile2=/etc/resolv.conf 2020-09-12T14:05:43.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:234: TEST_divergent_ec: local num_osds=3 2020-09-12T14:05:43.999 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:235: TEST_divergent_ec: expr 3 - 1 2020-09-12T14:05:43.999 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:235: TEST_divergent_ec: seq 0 2 2020-09-12T14:05:44.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:235: TEST_divergent_ec: local 'osds=0 2020-09-12T14:05:44.000 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:05:44.000 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T14:05:44.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:236: TEST_divergent_ec: run_mon td/divergent-priors a 2020-09-12T14:05:44.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/divergent-priors 2020-09-12T14:05:44.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2020-09-12T14:05:44.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2020-09-12T14:05:44.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2020-09-12T14:05:44.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/divergent-priors/a 2020-09-12T14:05:44.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2020-09-12T14:05:44.303 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2020-09-12T14:05:44.303 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:05:44.303 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:05:44.303 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:05:44.304 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:05:44.304 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:05:44.304 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:05:44.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 2020-09-12T14:05:44.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: cat 2020-09-12T14:05:44.489 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a fsid 2020-09-12T14:05:44.489 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:05:44.489 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:05:44.489 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=fsid 2020-09-12T14:05:44.490 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .fsid 2020-09-12T14:05:44.491 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:05:44.491 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:05:44.491 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:05:44.491 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:05:44.491 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:05:44.491 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:05:44.491 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-mon.a.asok 2020-09-12T14:05:44.492 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:05:44.492 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.20815/ceph-mon.a.asok config get fsid 2020-09-12T14:05:44.597 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a mon_host 2020-09-12T14:05:44.598 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:05:44.598 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:05:44.598 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=mon_host 2020-09-12T14:05:44.598 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .mon_host 2020-09-12T14:05:44.599 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:05:44.599 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:05:44.599 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:05:44.599 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:05:44.600 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:05:44.600 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:05:44.600 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-mon.a.asok 2020-09-12T14:05:44.600 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:05:44.600 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.20815/ceph-mon.a.asok config get mon_host 2020-09-12T14:05:44.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:237: TEST_divergent_ec: run_mgr td/divergent-priors x 2020-09-12T14:05:44.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: local dir=td/divergent-priors 2020-09-12T14:05:44.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: shift 2020-09-12T14:05:44.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: local id=x 2020-09-12T14:05:44.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: shift 2020-09-12T14:05:44.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: local data=td/divergent-priors/x 2020-09-12T14:05:44.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:558: run_mgr: ceph config set mgr mgr/devicehealth/enable_monitoring off --force 2020-09-12T14:05:45.610 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: get_asok_path 2020-09-12T14:05:45.610 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:05:45.611 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:05:45.611 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:05:45.611 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:05:45.611 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:05:45.611 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:05:45.612 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:05:45.612 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:05:45.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:238: TEST_divergent_ec: for i in $osds 2020-09-12T14:05:45.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:240: TEST_divergent_ec: run_osd td/divergent-priors 0 2020-09-12T14:05:45.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/divergent-priors 2020-09-12T14:05:45.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:05:45.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=0 2020-09-12T14:05:45.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:05:45.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/divergent-priors/0 2020-09-12T14:05:45.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T14:05:45.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:05:45.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:05:45.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:05:45.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2020-09-12T14:05:45.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2020-09-12T14:05:45.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:05:45.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:05:45.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T14:05:45.643 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:05:45.643 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:05:45.643 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:05:45.643 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:05:45.643 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:05:45.643 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:05:45.643 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:05:45.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:05:45.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:05:45.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:05:45.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:05:45.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T14:05:45.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T14:05:45.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:05:45.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:05:45.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:05:45.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:05:45.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:05:45.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/divergent-priors/0 2020-09-12T14:05:45.646 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:05:45.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=3d802381-3749-43a0-ac4c-e7fa2730ba5f 2020-09-12T14:05:45.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd0 3d802381-3749-43a0-ac4c-e7fa2730ba5f' 2020-09-12T14:05:45.647 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:05:45.647 INFO:tasks.workunit.client.0.smithi027.stdout:add osd0 3d802381-3749-43a0-ac4c-e7fa2730ba5f 2020-09-12T14:05:45.662 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQC51VxfIoxzJxAABlgibg4k6QSuPPl/gVx4lQ== 2020-09-12T14:05:45.663 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQC51VxfIoxzJxAABlgibg4k6QSuPPl/gVx4lQ=="}' 2020-09-12T14:05:45.663 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 3d802381-3749-43a0-ac4c-e7fa2730ba5f -i td/divergent-priors/0/new.json 2020-09-12T14:05:45.991 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:05:46.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/divergent-priors/0/new.json 2020-09-12T14:05:46.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 0 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQC51VxfIoxzJxAABlgibg4k6QSuPPl/gVx4lQ== --osd-uuid 3d802381-3749-43a0-ac4c-e7fa2730ba5f 2020-09-12T14:05:46.033 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:46.032+0000 7faf0efe7ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:05:46.035 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:46.034+0000 7faf0efe7ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:05:46.037 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:46.036+0000 7faf0efe7ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:05:46.037 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:46.037+0000 7faf0efe7ec0 -1 bluestore(td/divergent-priors/0/block) _read_bdev_label failed to open td/divergent-priors/0/block: (2) No such file or directory 2020-09-12T14:05:46.037 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:46.037+0000 7faf0efe7ec0 -1 bluestore(td/divergent-priors/0/block) _read_bdev_label failed to open td/divergent-priors/0/block: (2) No such file or directory 2020-09-12T14:05:46.038 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:46.037+0000 7faf0efe7ec0 -1 bluestore(td/divergent-priors/0/block) _read_bdev_label failed to open td/divergent-priors/0/block: (2) No such file or directory 2020-09-12T14:05:46.079 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:46.077+0000 7faf0efe7ec0 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2020-09-12T14:05:50.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/divergent-priors/0/keyring 2020-09-12T14:05:50.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:05:50.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd0 key to auth repository 2020-09-12T14:05:50.590 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:05:50.590 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd0 key to auth repository 2020-09-12T14:05:51.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.0 2020-09-12T14:05:51.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 0 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:05:51.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:05:51.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:05:51.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:05:51.150 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.0 2020-09-12T14:05:51.177 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:51.175+0000 7fe362294ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:05:51.178 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:51.177+0000 7fe362294ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:05:51.179 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:51.178+0000 7fe362294ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:05:51.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 0 2020-09-12T14:05:51.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:05:51.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T14:05:51.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:05:51.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:05:51.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:05:51.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:05:51.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:05:51.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:05:51.563 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:05:51.700 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:51.699+0000 7fe362294ec0 -1 Falling back to public interface 2020-09-12T14:05:51.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:05:52.453 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:52.451+0000 7fe362294ec0 -1 osd.0 0 log_to_monitors {default=true} 2020-09-12T14:05:52.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:05:52.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:05:52.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:05:52.970 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:05:52.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:05:52.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:05:53.397 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/54731,v1:127.0.0.1:6803/54731] [v2:127.0.0.1:6804/54731,v1:127.0.0.1:6805/54731] exists,up 3d802381-3749-43a0-ac4c-e7fa2730ba5f 2020-09-12T14:05:53.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:05:53.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:05:53.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:05:53.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:238: TEST_divergent_ec: for i in $osds 2020-09-12T14:05:53.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:240: TEST_divergent_ec: run_osd td/divergent-priors 1 2020-09-12T14:05:53.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/divergent-priors 2020-09-12T14:05:53.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:05:53.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=1 2020-09-12T14:05:53.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:05:53.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/divergent-priors/1 2020-09-12T14:05:53.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T14:05:53.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:05:53.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:05:53.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:05:53.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2020-09-12T14:05:53.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2020-09-12T14:05:53.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:05:53.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:05:53.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T14:05:53.400 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:05:53.400 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:05:53.401 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:05:53.401 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:05:53.401 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:05:53.401 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:05:53.401 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:05:53.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:05:53.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:05:53.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:05:53.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:05:53.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T14:05:53.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T14:05:53.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:05:53.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:05:53.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:05:53.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:05:53.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:05:53.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/divergent-priors/1 2020-09-12T14:05:53.403 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:05:53.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=270594ae-5b9c-4b0c-bd1d-3fddd2a993d0 2020-09-12T14:05:53.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd1 270594ae-5b9c-4b0c-bd1d-3fddd2a993d0' 2020-09-12T14:05:53.404 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:05:53.404 INFO:tasks.workunit.client.0.smithi027.stdout:add osd1 270594ae-5b9c-4b0c-bd1d-3fddd2a993d0 2020-09-12T14:05:53.418 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDB1VxfAIHjGBAAyxI7t1s1s26zdvQIvqBVTA== 2020-09-12T14:05:53.418 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDB1VxfAIHjGBAAyxI7t1s1s26zdvQIvqBVTA=="}' 2020-09-12T14:05:53.418 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 270594ae-5b9c-4b0c-bd1d-3fddd2a993d0 -i td/divergent-priors/1/new.json 2020-09-12T14:05:54.066 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:05:54.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/divergent-priors/1/new.json 2020-09-12T14:05:54.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 1 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDB1VxfAIHjGBAAyxI7t1s1s26zdvQIvqBVTA== --osd-uuid 270594ae-5b9c-4b0c-bd1d-3fddd2a993d0 2020-09-12T14:05:54.109 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:54.108+0000 7f65c181eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:05:54.111 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:54.109+0000 7f65c181eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:05:54.113 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:54.111+0000 7f65c181eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:05:54.113 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:54.112+0000 7f65c181eec0 -1 bluestore(td/divergent-priors/1/block) _read_bdev_label failed to open td/divergent-priors/1/block: (2) No such file or directory 2020-09-12T14:05:54.114 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:54.112+0000 7f65c181eec0 -1 bluestore(td/divergent-priors/1/block) _read_bdev_label failed to open td/divergent-priors/1/block: (2) No such file or directory 2020-09-12T14:05:54.114 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:54.112+0000 7f65c181eec0 -1 bluestore(td/divergent-priors/1/block) _read_bdev_label failed to open td/divergent-priors/1/block: (2) No such file or directory 2020-09-12T14:05:54.220 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:54.219+0000 7f65c181eec0 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2020-09-12T14:05:58.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/divergent-priors/1/keyring 2020-09-12T14:05:58.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:05:58.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd1 key to auth repository 2020-09-12T14:05:58.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:05:58.486 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd1 key to auth repository 2020-09-12T14:05:59.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.1 2020-09-12T14:05:59.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 1 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:05:59.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:05:59.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:05:59.037 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T14:05:59.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:05:59.064 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:59.063+0000 7f5dc4b98ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:05:59.066 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:59.064+0000 7f5dc4b98ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:05:59.068 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:59.066+0000 7f5dc4b98ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:05:59.454 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 1 2020-09-12T14:05:59.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:05:59.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:05:59.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:05:59.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:05:59.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:05:59.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:05:59.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:05:59.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:05:59.456 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:05:59.588 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:05:59.587+0000 7f5dc4b98ec0 -1 Falling back to public interface 2020-09-12T14:05:59.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:06:00.411 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:06:00.410+0000 7f5dc4b98ec0 -1 osd.1 0 log_to_monitors {default=true} 2020-09-12T14:06:00.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:06:00.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:06:00.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:06:00.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:06:00.901 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:06:00.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:06:01.355 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/55733,v1:127.0.0.1:6811/55733] [v2:127.0.0.1:6812/55733,v1:127.0.0.1:6813/55733] exists,up 270594ae-5b9c-4b0c-bd1d-3fddd2a993d0 2020-09-12T14:06:01.355 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:06:01.355 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:06:01.355 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:06:01.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:238: TEST_divergent_ec: for i in $osds 2020-09-12T14:06:01.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:240: TEST_divergent_ec: run_osd td/divergent-priors 2 2020-09-12T14:06:01.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/divergent-priors 2020-09-12T14:06:01.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:06:01.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=2 2020-09-12T14:06:01.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:06:01.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/divergent-priors/2 2020-09-12T14:06:01.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T14:06:01.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:06:01.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:06:01.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:06:01.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2020-09-12T14:06:01.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2020-09-12T14:06:01.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:06:01.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:06:01.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T14:06:01.358 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:06:01.358 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:06:01.358 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:06:01.358 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:06:01.358 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:06:01.358 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:06:01.359 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:06:01.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:06:01.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:06:01.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:06:01.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:06:01.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T14:06:01.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T14:06:01.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:06:01.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:06:01.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:06:01.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:06:01.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:06:01.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/divergent-priors/2 2020-09-12T14:06:01.361 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:06:01.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=ed2b134c-2770-4bad-8557-6c53d9f7b711 2020-09-12T14:06:01.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd2 ed2b134c-2770-4bad-8557-6c53d9f7b711' 2020-09-12T14:06:01.361 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:06:01.361 INFO:tasks.workunit.client.0.smithi027.stdout:add osd2 ed2b134c-2770-4bad-8557-6c53d9f7b711 2020-09-12T14:06:01.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDJ1Vxf9aZrFhAAAWj55HW9+EHTBfQLZFqN1Q== 2020-09-12T14:06:01.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDJ1Vxf9aZrFhAAAWj55HW9+EHTBfQLZFqN1Q=="}' 2020-09-12T14:06:01.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new ed2b134c-2770-4bad-8557-6c53d9f7b711 -i td/divergent-priors/2/new.json 2020-09-12T14:06:01.899 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:06:01.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/divergent-priors/2/new.json 2020-09-12T14:06:01.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 2 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDJ1Vxf9aZrFhAAAWj55HW9+EHTBfQLZFqN1Q== --osd-uuid ed2b134c-2770-4bad-8557-6c53d9f7b711 2020-09-12T14:06:01.942 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:06:01.940+0000 7efc0767bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:06:01.943 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:06:01.942+0000 7efc0767bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:06:01.945 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:06:01.944+0000 7efc0767bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:06:01.946 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:06:01.944+0000 7efc0767bec0 -1 bluestore(td/divergent-priors/2/block) _read_bdev_label failed to open td/divergent-priors/2/block: (2) No such file or directory 2020-09-12T14:06:01.946 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:06:01.944+0000 7efc0767bec0 -1 bluestore(td/divergent-priors/2/block) _read_bdev_label failed to open td/divergent-priors/2/block: (2) No such file or directory 2020-09-12T14:06:01.946 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:06:01.944+0000 7efc0767bec0 -1 bluestore(td/divergent-priors/2/block) _read_bdev_label failed to open td/divergent-priors/2/block: (2) No such file or directory 2020-09-12T14:06:01.976 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:06:01.975+0000 7efc0767bec0 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2020-09-12T14:06:06.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/divergent-priors/2/keyring 2020-09-12T14:06:06.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:06:06.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd2 key to auth repository 2020-09-12T14:06:06.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:06:06.629 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd2 key to auth repository 2020-09-12T14:06:07.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.2 2020-09-12T14:06:07.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 2 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:06:07.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:06:07.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:06:07.185 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.2 2020-09-12T14:06:07.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:06:07.214 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:06:07.213+0000 7f9e4935eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:06:07.232 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:06:07.231+0000 7f9e4935eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:06:07.233 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:06:07.232+0000 7f9e4935eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:06:07.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 2 2020-09-12T14:06:07.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:06:07.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T14:06:07.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:06:07.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:06:07.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:06:07.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:06:07.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:06:07.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:06:07.595 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:06:07.748 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:06:07.748+0000 7f9e4935eec0 -1 Falling back to public interface 2020-09-12T14:06:07.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:06:08.517 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:06:08.516+0000 7f9e4935eec0 -1 osd.2 0 log_to_monitors {default=true} 2020-09-12T14:06:09.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:06:09.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:06:09.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:06:09.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:06:09.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:06:09.002 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:06:09.429 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/56727,v1:127.0.0.1:6819/56727] [v2:127.0.0.1:6820/56727,v1:127.0.0.1:6821/56727] exists,up ed2b134c-2770-4bad-8557-6c53d9f7b711 2020-09-12T14:06:09.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:06:09.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:06:09.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:06:09.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:243: TEST_divergent_ec: ceph osd set noout 2020-09-12T14:06:10.223 INFO:tasks.workunit.client.0.smithi027.stderr:noout is set 2020-09-12T14:06:10.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:244: TEST_divergent_ec: ceph osd set noin 2020-09-12T14:06:11.000 INFO:tasks.workunit.client.0.smithi027.stderr:noin is set 2020-09-12T14:06:11.009 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:245: TEST_divergent_ec: ceph osd set nodown 2020-09-12T14:06:11.827 INFO:tasks.workunit.client.0.smithi027.stderr:nodown is set 2020-09-12T14:06:11.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:246: TEST_divergent_ec: create_ec_pool test true k=2 m=1 2020-09-12T14:06:11.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: create_ec_pool: local pool_name=test 2020-09-12T14:06:11.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: create_ec_pool: shift 2020-09-12T14:06:11.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: create_ec_pool: local allow_overwrites=true 2020-09-12T14:06:11.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: create_ec_pool: shift 2020-09-12T14:06:11.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 2020-09-12T14:06:12.481 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: create_ec_pool: create_pool test 1 1 erasure myprofile 2020-09-12T14:06:12.482 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test 1 1 erasure myprofile 2020-09-12T14:06:13.264 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test' already exists 2020-09-12T14:06:13.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:06:14.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2281: create_ec_pool: '[' true = true ']' 2020-09-12T14:06:14.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: create_ec_pool: ceph osd pool set test allow_ec_overwrites true 2020-09-12T14:06:15.179 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 1 allow_ec_overwrites to true 2020-09-12T14:06:15.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: create_ec_pool: wait_for_clean 2020-09-12T14:06:15.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:06:15.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:06:15.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:06:15.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:06:15.190 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 90 .1 2020-09-12T14:06:15.190 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:06:15.191 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:06:15.191 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:06:15.191 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:06:15.191 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:06:15.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:06:15.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:06:15.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:06:15.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:06:15.287 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:06:15.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:06:15.705 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:06:15.706 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T14:06:15.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:06:15.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:06:15.706 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:06:15.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836487 2020-09-12T14:06:15.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836487 2020-09-12T14:06:15.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836487' 2020-09-12T14:06:15.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:06:15.853 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:06:15.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672965 2020-09-12T14:06:15.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672965 2020-09-12T14:06:16.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836487 1-42949672965' 2020-09-12T14:06:16.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:06:16.000 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:06:16.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509445 2020-09-12T14:06:16.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509445 2020-09-12T14:06:16.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836487 1-42949672965 2-64424509445' 2020-09-12T14:06:16.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:06:16.147 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836487 2020-09-12T14:06:16.147 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:06:16.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:06:16.148 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836487 2020-09-12T14:06:16.148 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:06:16.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836487 2020-09-12T14:06:16.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2020-09-12T14:06:16.149 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836487 2020-09-12T14:06:16.150 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:06:16.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836485 -lt 21474836487 2020-09-12T14:06:16.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:06:17.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:06:17.580 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:06:18.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836487 -lt 21474836487 2020-09-12T14:06:18.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:06:18.030 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672965 2020-09-12T14:06:18.031 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:06:18.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:06:18.031 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672965 2020-09-12T14:06:18.032 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:06:18.032 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672965 2020-09-12T14:06:18.032 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672965' 2020-09-12T14:06:18.033 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672965 2020-09-12T14:06:18.033 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:06:18.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672966 -lt 42949672965 2020-09-12T14:06:18.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:06:18.444 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509445 2020-09-12T14:06:18.444 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:06:18.444 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:06:18.445 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509445 2020-09-12T14:06:18.445 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:06:18.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509445 2020-09-12T14:06:18.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509445' 2020-09-12T14:06:18.446 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509445 2020-09-12T14:06:18.447 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:06:18.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509445 -lt 64424509445 2020-09-12T14:06:18.859 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:06:18.859 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:06:18.859 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:06:19.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 1 == 0 2020-09-12T14:06:19.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:06:19.410 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:06:19.411 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:06:19.411 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:06:19.411 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:06:19.411 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:06:19.412 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:06:19.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:06:19.809 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:06:19.809 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:06:19.810 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:06:20.342 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 1 2020-09-12T14:06:20.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T14:06:20.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T14:06:20.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: create_ec_pool: return 0 2020-09-12T14:06:20.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:248: TEST_divergent_ec: flush_pg_stats 2020-09-12T14:06:20.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:06:20.343 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:06:20.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:06:20.749 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:06:20.750 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T14:06:20.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:06:20.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:06:20.750 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:06:20.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836490 2020-09-12T14:06:20.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836490 2020-09-12T14:06:20.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836490' 2020-09-12T14:06:20.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:06:20.897 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:06:21.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672968 2020-09-12T14:06:21.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672968 2020-09-12T14:06:21.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836490 1-42949672968' 2020-09-12T14:06:21.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:06:21.045 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:06:21.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509448 2020-09-12T14:06:21.191 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509448 2020-09-12T14:06:21.191 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836490 1-42949672968 2-64424509448' 2020-09-12T14:06:21.191 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:06:21.192 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836490 2020-09-12T14:06:21.192 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:06:21.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:06:21.192 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836490 2020-09-12T14:06:21.193 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:06:21.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836490 2020-09-12T14:06:21.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2020-09-12T14:06:21.194 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836490 2020-09-12T14:06:21.194 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:06:21.604 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836490 -lt 21474836490 2020-09-12T14:06:21.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:06:21.605 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672968 2020-09-12T14:06:21.605 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:06:21.606 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:06:21.606 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672968 2020-09-12T14:06:21.606 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:06:21.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672968 2020-09-12T14:06:21.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672968' 2020-09-12T14:06:21.608 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672968 2020-09-12T14:06:21.608 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:06:22.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672968 -lt 42949672968 2020-09-12T14:06:22.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:06:22.011 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509448 2020-09-12T14:06:22.011 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:06:22.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:06:22.012 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509448 2020-09-12T14:06:22.012 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:06:22.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509448 2020-09-12T14:06:22.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509448' 2020-09-12T14:06:22.014 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509448 2020-09-12T14:06:22.014 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:06:22.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509448 -lt 64424509448 2020-09-12T14:06:22.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:249: TEST_divergent_ec: wait_for_clean 2020-09-12T14:06:22.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:06:22.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:06:22.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:06:22.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:06:22.421 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 90 .1 2020-09-12T14:06:22.421 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:06:22.421 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:06:22.421 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:06:22.422 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:06:22.422 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:06:22.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:06:22.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:06:22.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:06:22.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:06:22.518 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:06:22.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:06:22.924 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:06:22.924 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T14:06:22.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:06:22.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:06:22.925 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:06:23.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836492 2020-09-12T14:06:23.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836492 2020-09-12T14:06:23.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836492' 2020-09-12T14:06:23.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:06:23.071 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:06:23.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672971 2020-09-12T14:06:23.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672971 2020-09-12T14:06:23.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836492 1-42949672971' 2020-09-12T14:06:23.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:06:23.218 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:06:23.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509450 2020-09-12T14:06:23.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509450 2020-09-12T14:06:23.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-64424509450' 2020-09-12T14:06:23.365 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:06:23.365 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836492 2020-09-12T14:06:23.365 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:06:23.365 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:06:23.366 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836492 2020-09-12T14:06:23.366 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:06:23.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836492 2020-09-12T14:06:23.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836492' 2020-09-12T14:06:23.367 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836492 2020-09-12T14:06:23.367 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:06:23.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836492 -lt 21474836492 2020-09-12T14:06:23.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:06:23.779 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672971 2020-09-12T14:06:23.780 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:06:23.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:06:23.780 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672971 2020-09-12T14:06:23.781 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:06:23.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672971 2020-09-12T14:06:23.782 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672971' 2020-09-12T14:06:23.782 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672971 2020-09-12T14:06:23.782 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:06:24.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672971 -lt 42949672971 2020-09-12T14:06:24.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:06:24.188 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509450 2020-09-12T14:06:24.188 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:06:24.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:06:24.189 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509450 2020-09-12T14:06:24.190 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:06:24.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509450 2020-09-12T14:06:24.191 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509450' 2020-09-12T14:06:24.191 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509450 2020-09-12T14:06:24.191 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:06:24.597 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509448 -lt 64424509450 2020-09-12T14:06:24.597 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:06:25.598 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:06:25.599 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:06:26.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509451 -lt 64424509450 2020-09-12T14:06:26.038 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:06:26.038 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:06:26.038 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:06:26.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 1 == 0 2020-09-12T14:06:26.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:06:26.577 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:06:26.578 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:06:26.578 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:06:26.578 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:06:26.578 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:06:26.579 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:06:26.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:06:26.973 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:06:26.973 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:06:26.973 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:06:27.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 1 2020-09-12T14:06:27.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T14:06:27.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T14:06:27.517 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:252: TEST_divergent_ec: ceph pg dump pgs --format=json 2020-09-12T14:06:27.517 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:252: TEST_divergent_ec: jq '.pg_stats[0].up_primary' 2020-09-12T14:06:27.907 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:06:27.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:252: TEST_divergent_ec: local divergent=1 2020-09-12T14:06:27.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:253: TEST_divergent_ec: echo 'primary and soon to be divergent is 1' 2020-09-12T14:06:27.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:254: TEST_divergent_ec: ceph pg dump pgs 2020-09-12T14:06:27.918 INFO:tasks.workunit.client.0.smithi027.stdout:primary and soon to be divergent is 1 2020-09-12T14:06:28.304 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:06:28.305 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 0 0 0 0 0 0 0 0 0 0 active+clean 2020-09-12T14:06:14.440221+0000 0'0 26:10 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:06:13.000609+0000 0'0 2020-09-12T14:06:13.000609+0000 0 2020-09-12T14:06:28.305 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:06:28.305 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:06:28.306 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:06:28.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:255: TEST_divergent_ec: local non_divergent= 2020-09-12T14:06:28.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:256: TEST_divergent_ec: for i in $osds 2020-09-12T14:06:28.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:258: TEST_divergent_ec: '[' 0 = 1 ']' 2020-09-12T14:06:28.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:261: TEST_divergent_ec: non_divergent=' 0' 2020-09-12T14:06:28.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:256: TEST_divergent_ec: for i in $osds 2020-09-12T14:06:28.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:258: TEST_divergent_ec: '[' 1 = 1 ']' 2020-09-12T14:06:28.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:259: TEST_divergent_ec: continue 2020-09-12T14:06:28.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:256: TEST_divergent_ec: for i in $osds 2020-09-12T14:06:28.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:258: TEST_divergent_ec: '[' 2 = 1 ']' 2020-09-12T14:06:28.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:261: TEST_divergent_ec: non_divergent=' 0 2' 2020-09-12T14:06:28.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:264: TEST_divergent_ec: echo 'writing initial objects' 2020-09-12T14:06:28.319 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: seq 1 100 2020-09-12T14:06:28.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:28.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_1 /etc/fstab 2020-09-12T14:06:28.320 INFO:tasks.workunit.client.0.smithi027.stdout:writing initial objects 2020-09-12T14:06:28.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:28.468 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_2 /etc/fstab 2020-09-12T14:06:28.677 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:28.678 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_3 /etc/fstab 2020-09-12T14:06:28.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:28.871 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_4 /etc/fstab 2020-09-12T14:06:29.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:29.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_5 /etc/fstab 2020-09-12T14:06:29.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:29.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_6 /etc/fstab 2020-09-12T14:06:29.551 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:29.551 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_7 /etc/fstab 2020-09-12T14:06:29.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:29.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_8 /etc/fstab 2020-09-12T14:06:29.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:29.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_9 /etc/fstab 2020-09-12T14:06:30.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:30.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_10 /etc/fstab 2020-09-12T14:06:30.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:30.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_11 /etc/fstab 2020-09-12T14:06:30.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:30.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_12 /etc/fstab 2020-09-12T14:06:30.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:30.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_13 /etc/fstab 2020-09-12T14:06:30.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:30.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_14 /etc/fstab 2020-09-12T14:06:31.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:31.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_15 /etc/fstab 2020-09-12T14:06:31.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:31.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_16 /etc/fstab 2020-09-12T14:06:31.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:31.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_17 /etc/fstab 2020-09-12T14:06:31.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:31.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_18 /etc/fstab 2020-09-12T14:06:32.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:32.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_19 /etc/fstab 2020-09-12T14:06:32.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:32.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_20 /etc/fstab 2020-09-12T14:06:32.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:32.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_21 /etc/fstab 2020-09-12T14:06:32.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:32.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_22 /etc/fstab 2020-09-12T14:06:32.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:32.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_23 /etc/fstab 2020-09-12T14:06:33.091 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:33.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_24 /etc/fstab 2020-09-12T14:06:33.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:33.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_25 /etc/fstab 2020-09-12T14:06:33.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:33.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_26 /etc/fstab 2020-09-12T14:06:33.771 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:33.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_27 /etc/fstab 2020-09-12T14:06:33.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:33.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_28 /etc/fstab 2020-09-12T14:06:34.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:34.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_29 /etc/fstab 2020-09-12T14:06:34.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:34.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_30 /etc/fstab 2020-09-12T14:06:34.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:34.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_31 /etc/fstab 2020-09-12T14:06:34.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:34.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_32 /etc/fstab 2020-09-12T14:06:35.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:35.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_33 /etc/fstab 2020-09-12T14:06:35.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:35.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_34 /etc/fstab 2020-09-12T14:06:35.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:35.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_35 /etc/fstab 2020-09-12T14:06:35.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:35.659 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_36 /etc/fstab 2020-09-12T14:06:35.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:35.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_37 /etc/fstab 2020-09-12T14:06:36.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:36.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_38 /etc/fstab 2020-09-12T14:06:36.246 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:36.246 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_39 /etc/fstab 2020-09-12T14:06:36.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:36.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_40 /etc/fstab 2020-09-12T14:06:36.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:36.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_41 /etc/fstab 2020-09-12T14:06:36.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:36.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_42 /etc/fstab 2020-09-12T14:06:37.084 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:37.084 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_43 /etc/fstab 2020-09-12T14:06:37.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:37.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_44 /etc/fstab 2020-09-12T14:06:37.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:37.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_45 /etc/fstab 2020-09-12T14:06:37.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:37.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_46 /etc/fstab 2020-09-12T14:06:37.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:37.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_47 /etc/fstab 2020-09-12T14:06:38.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:38.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_48 /etc/fstab 2020-09-12T14:06:38.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:38.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_49 /etc/fstab 2020-09-12T14:06:38.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:38.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_50 /etc/fstab 2020-09-12T14:06:38.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:38.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_51 /etc/fstab 2020-09-12T14:06:39.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:39.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_52 /etc/fstab 2020-09-12T14:06:39.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:39.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_53 /etc/fstab 2020-09-12T14:06:39.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:39.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_54 /etc/fstab 2020-09-12T14:06:39.669 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:39.669 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_55 /etc/fstab 2020-09-12T14:06:39.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:39.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_56 /etc/fstab 2020-09-12T14:06:40.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:40.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_57 /etc/fstab 2020-09-12T14:06:40.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:40.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_58 /etc/fstab 2020-09-12T14:06:40.508 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:40.508 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_59 /etc/fstab 2020-09-12T14:06:40.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:40.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_60 /etc/fstab 2020-09-12T14:06:40.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:40.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_61 /etc/fstab 2020-09-12T14:06:41.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:41.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_62 /etc/fstab 2020-09-12T14:06:41.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:41.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_63 /etc/fstab 2020-09-12T14:06:41.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:41.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_64 /etc/fstab 2020-09-12T14:06:41.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:41.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_65 /etc/fstab 2020-09-12T14:06:41.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:41.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_66 /etc/fstab 2020-09-12T14:06:42.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:42.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_67 /etc/fstab 2020-09-12T14:06:42.353 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:42.353 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_68 /etc/fstab 2020-09-12T14:06:42.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:42.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_69 /etc/fstab 2020-09-12T14:06:42.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:42.782 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_70 /etc/fstab 2020-09-12T14:06:42.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:42.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_71 /etc/fstab 2020-09-12T14:06:43.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:43.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_72 /etc/fstab 2020-09-12T14:06:43.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:43.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_73 /etc/fstab 2020-09-12T14:06:43.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:43.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_74 /etc/fstab 2020-09-12T14:06:43.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:43.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_75 /etc/fstab 2020-09-12T14:06:44.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:44.032 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_76 /etc/fstab 2020-09-12T14:06:44.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:44.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_77 /etc/fstab 2020-09-12T14:06:44.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:44.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_78 /etc/fstab 2020-09-12T14:06:44.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:44.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_79 /etc/fstab 2020-09-12T14:06:44.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:44.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_80 /etc/fstab 2020-09-12T14:06:45.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:45.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_81 /etc/fstab 2020-09-12T14:06:45.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:45.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_82 /etc/fstab 2020-09-12T14:06:45.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:45.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_83 /etc/fstab 2020-09-12T14:06:45.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:45.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_84 /etc/fstab 2020-09-12T14:06:45.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:45.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_85 /etc/fstab 2020-09-12T14:06:46.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:46.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_86 /etc/fstab 2020-09-12T14:06:46.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:46.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_87 /etc/fstab 2020-09-12T14:06:46.672 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:46.673 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_88 /etc/fstab 2020-09-12T14:06:46.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:46.908 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_89 /etc/fstab 2020-09-12T14:06:47.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:47.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_90 /etc/fstab 2020-09-12T14:06:47.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:47.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_91 /etc/fstab 2020-09-12T14:06:47.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:47.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_92 /etc/fstab 2020-09-12T14:06:47.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:47.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_93 /etc/fstab 2020-09-12T14:06:47.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:47.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_94 /etc/fstab 2020-09-12T14:06:48.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:48.116 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_95 /etc/fstab 2020-09-12T14:06:48.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:48.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_96 /etc/fstab 2020-09-12T14:06:48.468 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:48.468 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_97 /etc/fstab 2020-09-12T14:06:48.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:48.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_98 /etc/fstab 2020-09-12T14:06:48.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:48.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_99 /etc/fstab 2020-09-12T14:06:49.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2020-09-12T14:06:49.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_100 /etc/fstab 2020-09-12T14:06:49.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:271: TEST_divergent_ec: WAIT_FOR_CLEAN_TIMEOUT=20 2020-09-12T14:06:49.291 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:271: TEST_divergent_ec: wait_for_clean 2020-09-12T14:06:49.291 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:06:49.291 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:06:49.291 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:06:49.291 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:06:49.292 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 20 .1 2020-09-12T14:06:49.292 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:06:49.292 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:06:49.292 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:06:49.293 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:06:49.293 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:06:49.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:06:49.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:06:49.362 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:06:49.362 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:06:49.362 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:06:49.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:06:49.773 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:06:49.773 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T14:06:49.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:06:49.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:06:49.773 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:06:49.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836500 2020-09-12T14:06:49.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836500 2020-09-12T14:06:49.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836500' 2020-09-12T14:06:49.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:06:49.925 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:06:50.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672978 2020-09-12T14:06:50.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672978 2020-09-12T14:06:50.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836500 1-42949672978' 2020-09-12T14:06:50.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:06:50.072 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:06:50.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509458 2020-09-12T14:06:50.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509458 2020-09-12T14:06:50.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836500 1-42949672978 2-64424509458' 2020-09-12T14:06:50.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:06:50.219 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836500 2020-09-12T14:06:50.219 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:06:50.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:06:50.220 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836500 2020-09-12T14:06:50.220 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:06:50.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836500 2020-09-12T14:06:50.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836500' 2020-09-12T14:06:50.221 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836500 2020-09-12T14:06:50.222 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:06:50.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836498 -lt 21474836500 2020-09-12T14:06:50.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:06:51.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:06:51.631 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:06:52.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836500 -lt 21474836500 2020-09-12T14:06:52.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:06:52.053 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672978 2020-09-12T14:06:52.054 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:06:52.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:06:52.055 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672978 2020-09-12T14:06:52.055 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:06:52.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672978 2020-09-12T14:06:52.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672978' 2020-09-12T14:06:52.055 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:06:52.056 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672978 2020-09-12T14:06:52.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672978 -lt 42949672978 2020-09-12T14:06:52.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:06:52.462 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509458 2020-09-12T14:06:52.463 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:06:52.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:06:52.463 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509458 2020-09-12T14:06:52.464 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:06:52.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509458 2020-09-12T14:06:52.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509458' 2020-09-12T14:06:52.465 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:06:52.465 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509458 2020-09-12T14:06:52.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509458 -lt 64424509458 2020-09-12T14:06:52.876 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:06:52.876 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:06:52.877 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:06:53.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 1 == 0 2020-09-12T14:06:53.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:06:53.431 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:06:53.431 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:06:53.431 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:06:53.431 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:06:53.431 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:06:53.431 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:06:53.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:06:53.827 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:06:53.827 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:06:53.827 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:06:54.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 1 2020-09-12T14:06:54.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T14:06:54.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T14:06:54.361 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:273: TEST_divergent_ec: get_pg test existing_1 2020-09-12T14:06:54.362 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1031: get_pg: local poolname=test 2020-09-12T14:06:54.362 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1032: get_pg: local objectname=existing_1 2020-09-12T14:06:54.362 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1034: get_pg: ceph --format json osd map test existing_1 2020-09-12T14:06:54.362 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1034: get_pg: jq -r .pgid 2020-09-12T14:06:54.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:273: TEST_divergent_ec: local pgid=1.0 2020-09-12T14:06:54.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:276: TEST_divergent_ec: echo 'blackholing osds 0 2' 2020-09-12T14:06:54.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:277: TEST_divergent_ec: ceph pg dump pgs 2020-09-12T14:06:54.774 INFO:tasks.workunit.client.0.smithi027.stdout:blackholing osds 0 2 2020-09-12T14:06:55.156 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:06:55.157 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 0 0 0 0 48300 0 0 16 16 active+clean 2020-09-12T14:06:14.440221+0000 29'100 29:110 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:06:13.000609+0000 0'0 2020-09-12T14:06:13.000609+0000 0 2020-09-12T14:06:55.157 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:06:55.157 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:06:55.158 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:06:55.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:278: TEST_divergent_ec: for i in $non_divergent 2020-09-12T14:06:55.168 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: get_asok_path osd.0 2020-09-12T14:06:55.168 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2020-09-12T14:06:55.168 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2020-09-12T14:06:55.169 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:06:55.169 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:06:55.169 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:06:55.169 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.0.asok 2020-09-12T14:06:55.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: CEPH_ARGS= 2020-09-12T14:06:55.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.0.asok config set objectstore_blackhole 1 2020-09-12T14:06:55.270 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:06:55.270 INFO:tasks.workunit.client.0.smithi027.stdout: "success": "objectstore_blackhole = 'true' (not observed, change may require restart) " 2020-09-12T14:06:55.270 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:06:55.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:278: TEST_divergent_ec: for i in $non_divergent 2020-09-12T14:06:55.278 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: get_asok_path osd.2 2020-09-12T14:06:55.279 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2020-09-12T14:06:55.279 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2020-09-12T14:06:55.279 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:06:55.280 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:06:55.280 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:06:55.280 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.2.asok 2020-09-12T14:06:55.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: CEPH_ARGS= 2020-09-12T14:06:55.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.2.asok config set objectstore_blackhole 1 2020-09-12T14:06:55.381 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:06:55.381 INFO:tasks.workunit.client.0.smithi027.stdout: "success": "objectstore_blackhole = 'true' (not observed, change may require restart) " 2020-09-12T14:06:55.381 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:06:55.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:284: TEST_divergent_ec: echo 'writing divergent object' 2020-09-12T14:06:55.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:286: TEST_divergent_ec: sleep 1 2020-09-12T14:06:55.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:285: TEST_divergent_ec: rados -p test put existing_100 /etc/resolv.conf 2020-09-12T14:06:55.390 INFO:tasks.workunit.client.0.smithi027.stdout:writing divergent object 2020-09-12T14:06:56.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:288: TEST_divergent_ec: rados -p test mksnap snap1 2020-09-12T14:06:56.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:287: TEST_divergent_ec: rados -p test put existing_100 /etc/fstab 2020-09-12T14:06:56.615 INFO:tasks.workunit.client.0.smithi027.stdout:created pool test snap snap1 2020-09-12T14:06:56.617 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:290: TEST_divergent_ec: sleep 10 2020-09-12T14:06:56.618 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:289: TEST_divergent_ec: expr 100 - 1 2020-09-12T14:06:56.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:289: TEST_divergent_ec: rados -p test put existing_99 /etc/fstab 2020-09-12T14:07:06.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:291: TEST_divergent_ec: killall -9 rados 2020-09-12T14:07:06.623 INFO:tasks.workunit.client.0.smithi027.stderr:rados: no process found 2020-09-12T14:07:06.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:294: TEST_divergent_ec: echo 'killing all the osds' 2020-09-12T14:07:06.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:295: TEST_divergent_ec: ceph pg dump pgs 2020-09-12T14:07:06.623 INFO:tasks.workunit.client.0.smithi027.stdout:killing all the osds 2020-09-12T14:07:07.027 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:07:07.028 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 101 0 0 0 0 48783 0 0 14 14 active+clean 2020-09-12T14:06:14.440221+0000 30'104 30:114 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:06:13.000609+0000 0'0 2020-09-12T14:06:13.000609+0000 0 2020-09-12T14:07:07.028 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:07:07.028 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:07:07.029 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:07:07.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:296: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd 2020-09-12T14:07:07.039 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:07:07.040 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:07:07.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:07:07.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:07:07.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:07:07.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:07:07.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:297: TEST_divergent_ec: for i in $osds 2020-09-12T14:07:07.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:299: TEST_divergent_ec: ceph osd down osd.0 2020-09-12T14:07:07.792 INFO:tasks.workunit.client.0.smithi027.stderr:osd.0 is already down. 2020-09-12T14:07:07.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:297: TEST_divergent_ec: for i in $osds 2020-09-12T14:07:07.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:299: TEST_divergent_ec: ceph osd down osd.1 2020-09-12T14:07:08.359 INFO:tasks.workunit.client.0.smithi027.stderr:osd.1 is already down. 2020-09-12T14:07:08.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:297: TEST_divergent_ec: for i in $osds 2020-09-12T14:07:08.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:299: TEST_divergent_ec: ceph osd down osd.2 2020-09-12T14:07:08.928 INFO:tasks.workunit.client.0.smithi027.stderr:osd.2 is already down. 2020-09-12T14:07:08.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:301: TEST_divergent_ec: for i in $non_divergent 2020-09-12T14:07:08.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:303: TEST_divergent_ec: ceph osd out osd.0 2020-09-12T14:07:09.495 INFO:tasks.workunit.client.0.smithi027.stderr:osd.0 is already out. 2020-09-12T14:07:09.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:301: TEST_divergent_ec: for i in $non_divergent 2020-09-12T14:07:09.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:303: TEST_divergent_ec: ceph osd out osd.2 2020-09-12T14:07:10.153 INFO:tasks.workunit.client.0.smithi027.stderr:osd.2 is already out. 2020-09-12T14:07:10.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:307: TEST_divergent_ec: echo 'bringing up non_divergent 0 2' 2020-09-12T14:07:10.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:308: TEST_divergent_ec: ceph pg dump pgs 2020-09-12T14:07:10.163 INFO:tasks.workunit.client.0.smithi027.stdout:bringing up non_divergent 0 2 2020-09-12T14:07:10.549 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:07:10.549 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 101 0 0 0 0 48783 0 0 14 14 stale+active+clean 2020-09-12T14:06:14.440221+0000 30'104 30:114 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:06:13.000609+0000 0'0 2020-09-12T14:06:13.000609+0000 0 2020-09-12T14:07:10.549 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:07:10.549 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:07:10.551 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:07:10.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:309: TEST_divergent_ec: for i in $non_divergent 2020-09-12T14:07:10.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:311: TEST_divergent_ec: activate_osd td/divergent-priors 0 2020-09-12T14:07:10.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/divergent-priors 2020-09-12T14:07:10.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T14:07:10.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=0 2020-09-12T14:07:10.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T14:07:10.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/divergent-priors/0 2020-09-12T14:07:10.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T14:07:10.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:07:10.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:07:10.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:07:10.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2020-09-12T14:07:10.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2020-09-12T14:07:10.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T14:07:10.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T14:07:10.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T14:07:10.565 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T14:07:10.565 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:07:10.565 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:07:10.565 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:07:10.565 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:07:10.565 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:07:10.566 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:07:10.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:07:10.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:07:10.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T14:07:10.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T14:07:10.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:07:10.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:07:10.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:07:10.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T14:07:10.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T14:07:10.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/divergent-priors/0 2020-09-12T14:07:10.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.0 2020-09-12T14:07:10.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 0 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:07:10.568 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/divergent-priors/0/whoami 2020-09-12T14:07:10.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 0 = 0 ']' 2020-09-12T14:07:10.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T14:07:10.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T14:07:10.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T14:07:10.569 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.0 2020-09-12T14:07:10.592 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:07:10.591+0000 7f4f8c810ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:07:10.594 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:07:10.592+0000 7f4f8c810ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:07:10.595 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:07:10.593+0000 7f4f8c810ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:07:10.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: wait_for_osd up 0 2020-09-12T14:07:10.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:07:10.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T14:07:10.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:07:10.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:07:10.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:07:10.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:07:10.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:07:10.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:07:10.979 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:07:11.114 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:07:11.113+0000 7f4f8c810ec0 -1 Falling back to public interface 2020-09-12T14:07:11.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:07:12.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:07:12.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:07:12.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:07:12.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:07:12.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:07:12.391 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:07:12.437 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:07:12.435+0000 7f4f8c810ec0 -1 osd.0 29 log_to_monitors {default=true} 2020-09-12T14:07:12.867 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up out weight 0 up_from 36 up_thru 0 down_at 31 last_clean_interval [5,29) [v2:127.0.0.1:6802/60986,v1:127.0.0.1:6803/60986] [v2:127.0.0.1:6804/60986,v1:127.0.0.1:6805/60986] exists,up 3d802381-3749-43a0-ac4c-e7fa2730ba5f 2020-09-12T14:07:12.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:07:12.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:07:12.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:07:12.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:309: TEST_divergent_ec: for i in $non_divergent 2020-09-12T14:07:12.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:311: TEST_divergent_ec: activate_osd td/divergent-priors 2 2020-09-12T14:07:12.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/divergent-priors 2020-09-12T14:07:12.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T14:07:12.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=2 2020-09-12T14:07:12.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T14:07:12.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/divergent-priors/2 2020-09-12T14:07:12.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T14:07:12.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:07:12.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:07:12.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:07:12.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2020-09-12T14:07:12.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2020-09-12T14:07:12.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T14:07:12.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T14:07:12.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T14:07:12.871 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T14:07:12.871 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:07:12.871 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:07:12.871 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:07:12.871 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:07:12.871 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:07:12.871 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:07:12.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:07:12.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:07:12.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T14:07:12.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T14:07:12.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:07:12.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:07:12.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:07:12.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T14:07:12.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T14:07:12.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/divergent-priors/2 2020-09-12T14:07:12.874 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.2 2020-09-12T14:07:12.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.2 2020-09-12T14:07:12.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 2 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:07:12.874 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/divergent-priors/2/whoami 2020-09-12T14:07:12.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 2 = 2 ']' 2020-09-12T14:07:12.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T14:07:12.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T14:07:12.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T14:07:12.897 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:07:12.895+0000 7f4c2aad9ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:07:12.898 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:07:12.897+0000 7f4c2aad9ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:07:12.899 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:07:12.898+0000 7f4c2aad9ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:07:13.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: wait_for_osd up 2 2020-09-12T14:07:13.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:07:13.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T14:07:13.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:07:13.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:07:13.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:07:13.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:07:13.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:07:13.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:07:13.287 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:07:13.417 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:07:13.415+0000 7f4c2aad9ec0 -1 Falling back to public interface 2020-09-12T14:07:13.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:07:14.604 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:07:14.602+0000 7f4c2aad9ec0 -1 osd.2 29 log_to_monitors {default=true} 2020-09-12T14:07:14.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:07:14.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:07:14.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:07:14.704 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:07:14.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:07:14.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:07:15.138 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up out weight 0 up_from 37 up_thru 0 down_at 33 last_clean_interval [15,29) [v2:127.0.0.1:6810/61385,v1:127.0.0.1:6811/61385] [v2:127.0.0.1:6812/61385,v1:127.0.0.1:6813/61385] exists,up ed2b134c-2770-4bad-8557-6c53d9f7b711 2020-09-12T14:07:15.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:07:15.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:07:15.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:07:15.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:313: TEST_divergent_ec: for i in $non_divergent 2020-09-12T14:07:15.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:315: TEST_divergent_ec: ceph osd in osd.0 2020-09-12T14:07:15.720 INFO:tasks.workunit.client.0.smithi027.stderr:osd.0 is already in. 2020-09-12T14:07:15.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:313: TEST_divergent_ec: for i in $non_divergent 2020-09-12T14:07:15.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:315: TEST_divergent_ec: ceph osd in osd.2 2020-09-12T14:07:16.292 INFO:tasks.workunit.client.0.smithi027.stderr:osd.2 is already in. 2020-09-12T14:07:16.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:318: TEST_divergent_ec: sleep 5 2020-09-12T14:07:21.304 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:322: TEST_divergent_ec: expr 2 + 2 2020-09-12T14:07:21.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:322: TEST_divergent_ec: objname=existing_4 2020-09-12T14:07:21.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:323: TEST_divergent_ec: echo 'writing non-divergent object existing_4' 2020-09-12T14:07:21.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:324: TEST_divergent_ec: ceph pg dump pgs 2020-09-12T14:07:21.305 INFO:tasks.workunit.client.0.smithi027.stdout:writing non-divergent object existing_4 2020-09-12T14:07:21.705 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:07:21.705 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 100 0 100 0 0 48300 0 0 16 16 active+undersized+degraded 2020-09-12T14:07:16.698987+0000 29'100 40:119 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2020-09-12T14:06:13.000609+0000 0'0 2020-09-12T14:06:13.000609+0000 0 2020-09-12T14:07:21.705 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:07:21.706 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:07:21.707 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:07:21.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:325: TEST_divergent_ec: rados -p test put existing_4 /etc/resolv.conf 2020-09-12T14:07:21.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:327: TEST_divergent_ec: WAIT_FOR_CLEAN_TIMEOUT=20 2020-09-12T14:07:21.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:327: TEST_divergent_ec: wait_for_clean 2020-09-12T14:07:21.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:07:21.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:07:21.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:07:21.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:07:21.938 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 20 .1 2020-09-12T14:07:21.938 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:07:21.938 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:07:21.939 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:07:21.939 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:07:21.939 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:07:21.997 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:07:21.997 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:07:21.997 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:07:21.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:07:21.998 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:07:22.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:07:22.412 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:07:22.412 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T14:07:22.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:07:22.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:07:22.413 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:07:22.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=154618822661 2020-09-12T14:07:22.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 154618822661 2020-09-12T14:07:22.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-154618822661' 2020-09-12T14:07:22.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:07:22.588 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:07:22.730 INFO:tasks.workunit.client.0.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-09-12T14:07:22.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq= 2020-09-12T14:07:22.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z '' 2020-09-12T14:07:22.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2075: flush_pg_stats: continue 2020-09-12T14:07:22.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:07:22.732 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:07:22.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=158913789956 2020-09-12T14:07:22.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 158913789956 2020-09-12T14:07:22.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-154618822661 2-158913789956' 2020-09-12T14:07:22.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:07:22.880 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-154618822661 2020-09-12T14:07:22.880 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:07:22.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:07:22.880 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-154618822661 2020-09-12T14:07:22.881 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:07:22.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=154618822661 2020-09-12T14:07:22.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 154618822661' 2020-09-12T14:07:22.882 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 154618822661 2020-09-12T14:07:22.882 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:07:23.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 154618822658 -lt 154618822661 2020-09-12T14:07:23.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:07:24.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:07:24.293 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:07:24.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 154618822661 -lt 154618822661 2020-09-12T14:07:24.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:07:24.713 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-158913789956 2020-09-12T14:07:24.713 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:07:24.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:07:24.714 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-158913789956 2020-09-12T14:07:24.715 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:07:24.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=158913789956 2020-09-12T14:07:24.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 158913789956' 2020-09-12T14:07:24.716 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:07:24.716 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 158913789956 2020-09-12T14:07:25.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 158913789956 -lt 158913789956 2020-09-12T14:07:25.154 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:07:25.155 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:07:25.155 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:07:25.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 1 == 0 2020-09-12T14:07:25.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:07:25.696 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:07:25.696 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:07:25.696 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:07:25.696 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:07:25.696 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:07:25.697 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:07:26.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:07:26.095 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:07:26.096 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:07:26.096 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:07:26.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:07:26.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' -1 2020-09-12T14:07:26.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1598: wait_for_clean: loop=0 2020-09-12T14:07:26.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1599: wait_for_clean: num_active_clean=0 2020-09-12T14:07:26.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:07:26.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:07:26.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:07:26.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:07:26.735 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:07:26.735 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:07:26.735 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:07:26.735 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:07:26.735 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:07:26.736 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:07:27.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:07:27.136 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:07:27.136 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:07:27.136 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:07:27.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:07:27.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:07:27.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:07:27.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:07:27.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:07:27.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:07:27.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:07:27.691 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:07:27.691 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:07:28.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:07:28.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:07:28.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 1 >= 8 )) 2020-09-12T14:07:28.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:07:28.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.2 2020-09-12T14:07:28.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:07:28.422 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:07:28.422 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:07:28.422 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:07:28.422 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:07:28.422 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:07:28.422 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:07:28.423 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:07:28.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:07:28.825 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:07:28.826 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:07:28.826 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:07:29.362 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:07:29.362 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:07:29.362 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:07:29.363 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:07:29.363 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:07:29.363 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:07:29.363 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:07:29.363 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:07:29.364 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:07:29.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:07:29.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:07:29.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 2 >= 8 )) 2020-09-12T14:07:29.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:07:29.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.4 2020-09-12T14:07:30.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:07:30.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:07:30.340 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:07:30.340 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:07:30.340 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:07:30.340 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:07:30.340 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:07:30.341 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:07:30.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:07:30.744 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:07:30.744 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:07:30.745 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:07:31.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:07:31.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:07:31.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:07:31.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:07:31.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:07:31.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:07:31.284 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:07:31.284 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:07:31.284 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:07:31.862 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:07:31.862 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:07:31.862 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 3 >= 8 )) 2020-09-12T14:07:31.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:07:31.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.8 2020-09-12T14:07:32.663 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:07:32.663 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:07:32.665 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:07:32.665 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:07:32.665 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:07:32.665 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:07:32.666 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:07:32.666 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:07:33.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:07:33.074 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:07:33.074 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:07:33.075 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:07:33.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:07:33.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:07:33.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:07:33.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:07:33.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:07:33.616 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:07:33.616 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:07:33.616 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:07:33.616 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:07:34.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:07:34.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:07:34.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 4 >= 8 )) 2020-09-12T14:07:34.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:07:34.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 1.6 2020-09-12T14:07:35.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:07:35.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:07:35.755 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:07:35.755 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:07:35.755 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:07:35.756 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:07:35.756 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:07:35.756 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:07:36.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:07:36.167 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:07:36.168 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:07:36.168 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:07:36.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:07:36.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:07:36.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:07:36.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:07:36.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:07:36.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:07:36.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:07:36.704 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:07:36.704 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:07:37.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:07:37.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:07:37.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 5 >= 8 )) 2020-09-12T14:07:37.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:07:37.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 3.2 2020-09-12T14:07:40.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:07:40.444 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:07:40.445 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:07:40.445 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:07:40.445 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:07:40.445 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:07:40.446 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:07:40.446 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:07:40.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:07:40.856 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:07:40.856 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:07:40.857 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:07:41.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:07:41.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:07:41.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:07:41.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:07:41.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:07:41.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:07:41.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:07:41.401 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:07:41.401 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:07:41.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:07:41.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:07:41.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 6 >= 8 )) 2020-09-12T14:07:41.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:07:41.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 6.4 2020-09-12T14:07:48.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:07:48.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:07:48.338 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:07:48.338 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:07:48.338 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:07:48.338 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:07:48.339 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:07:48.339 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:07:48.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:07:48.745 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:07:48.746 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:07:48.746 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:07:49.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:07:49.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:07:49.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:07:49.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:07:49.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:07:49.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:07:49.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:07:49.276 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:07:49.276 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:07:49.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:07:49.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:07:49.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 7 >= 8 )) 2020-09-12T14:07:49.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:07:49.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 7.3 2020-09-12T14:07:57.113 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:07:57.113 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:07:57.114 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:07:57.114 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:07:57.114 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:07:57.114 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:07:57.115 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:07:57.115 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:07:57.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:07:57.522 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:07:57.522 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:07:57.522 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:07:58.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:07:58.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:07:58.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:07:58.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:07:58.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:07:58.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:07:58.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:07:58.058 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:07:58.059 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:07:58.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:07:58.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:07:58.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 8 >= 8 )) 2020-09-12T14:07:58.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: wait_for_clean: ceph report 2020-09-12T14:07:59.116 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:07:59.116 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_fingerprint": "d3487fa8-4c06-46da-bbe3-8d05ca5a9357", 2020-09-12T14:07:59.116 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:07:59.116 INFO:tasks.workunit.client.0.smithi027.stdout: "commit": "f346cc73d15a568d3bec1c99a58953de76f57b85", 2020-09-12T14:07:59.116 INFO:tasks.workunit.client.0.smithi027.stdout: "timestamp": "2020-09-12T14:07:59.114615+0000", 2020-09-12T14:07:59.117 INFO:tasks.workunit.client.0.smithi027.stdout: "tag": "", 2020-09-12T14:07:59.117 INFO:tasks.workunit.client.0.smithi027.stdout: "health": { 2020-09-12T14:07:59.117 INFO:tasks.workunit.client.0.smithi027.stdout: "status": "HEALTH_WARN", 2020-09-12T14:07:59.117 INFO:tasks.workunit.client.0.smithi027.stdout: "checks": { 2020-09-12T14:07:59.117 INFO:tasks.workunit.client.0.smithi027.stdout: "OSDMAP_FLAGS": { 2020-09-12T14:07:59.117 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:07:59.117 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:07:59.118 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "nodown,noout,noin flag(s) set", 2020-09-12T14:07:59.118 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 17 2020-09-12T14:07:59.118 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.118 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [], 2020-09-12T14:07:59.118 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:07:59.118 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.118 INFO:tasks.workunit.client.0.smithi027.stdout: "OSD_DOWN": { 2020-09-12T14:07:59.119 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:07:59.119 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:07:59.119 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "1 osds down", 2020-09-12T14:07:59.119 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 1 2020-09-12T14:07:59.119 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.119 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [ 2020-09-12T14:07:59.119 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.119 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "osd.1 (root=default,host=smithi027) is down" 2020-09-12T14:07:59.120 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.120 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:07:59.120 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:07:59.120 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.120 INFO:tasks.workunit.client.0.smithi027.stdout: "PG_DEGRADED": { 2020-09-12T14:07:59.120 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:07:59.120 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:07:59.121 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "Degraded data redundancy: 101/303 objects degraded (33.333%), 1 pg degraded", 2020-09-12T14:07:59.121 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 1 2020-09-12T14:07:59.121 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.121 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [ 2020-09-12T14:07:59.121 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.121 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [2147483647,0,2]" 2020-09-12T14:07:59.121 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.121 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:07:59.122 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:07:59.122 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.122 INFO:tasks.workunit.client.0.smithi027.stdout: "POOL_APP_NOT_ENABLED": { 2020-09-12T14:07:59.122 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:07:59.122 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:07:59.122 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "1 pool(s) do not have an application enabled", 2020-09-12T14:07:59.122 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 1 2020-09-12T14:07:59.123 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.123 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [ 2020-09-12T14:07:59.123 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.123 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "application not enabled on pool 'test'" 2020-09-12T14:07:59.123 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.123 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.123 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2020-09-12T14:07:59.123 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.124 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:07:59.124 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:07:59.124 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.124 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.124 INFO:tasks.workunit.client.0.smithi027.stdout: "mutes": [] 2020-09-12T14:07:59.124 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.124 INFO:tasks.workunit.client.0.smithi027.stdout: "monmap_first_committed": 1, 2020-09-12T14:07:59.125 INFO:tasks.workunit.client.0.smithi027.stdout: "monmap_last_committed": 1, 2020-09-12T14:07:59.125 INFO:tasks.workunit.client.0.smithi027.stdout: "monmap": { 2020-09-12T14:07:59.125 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 1, 2020-09-12T14:07:59.125 INFO:tasks.workunit.client.0.smithi027.stdout: "fsid": "7e90bad3-7973-4a8c-be65-6326ab931fe8", 2020-09-12T14:07:59.125 INFO:tasks.workunit.client.0.smithi027.stdout: "modified": "2020-09-12T14:05:44.022891Z", 2020-09-12T14:07:59.125 INFO:tasks.workunit.client.0.smithi027.stdout: "created": "2020-09-12T14:05:44.022891Z", 2020-09-12T14:07:59.125 INFO:tasks.workunit.client.0.smithi027.stdout: "min_mon_release": 16, 2020-09-12T14:07:59.125 INFO:tasks.workunit.client.0.smithi027.stdout: "min_mon_release_name": "pacific", 2020-09-12T14:07:59.126 INFO:tasks.workunit.client.0.smithi027.stdout: "features": { 2020-09-12T14:07:59.126 INFO:tasks.workunit.client.0.smithi027.stdout: "persistent": [ 2020-09-12T14:07:59.126 INFO:tasks.workunit.client.0.smithi027.stdout: "kraken", 2020-09-12T14:07:59.126 INFO:tasks.workunit.client.0.smithi027.stdout: "luminous", 2020-09-12T14:07:59.126 INFO:tasks.workunit.client.0.smithi027.stdout: "mimic", 2020-09-12T14:07:59.126 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap-prune", 2020-09-12T14:07:59.126 INFO:tasks.workunit.client.0.smithi027.stdout: "nautilus", 2020-09-12T14:07:59.127 INFO:tasks.workunit.client.0.smithi027.stdout: "octopus", 2020-09-12T14:07:59.127 INFO:tasks.workunit.client.0.smithi027.stdout: "pacific" 2020-09-12T14:07:59.127 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:07:59.127 INFO:tasks.workunit.client.0.smithi027.stdout: "optional": [] 2020-09-12T14:07:59.127 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.127 INFO:tasks.workunit.client.0.smithi027.stdout: "mons": [ 2020-09-12T14:07:59.127 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.127 INFO:tasks.workunit.client.0.smithi027.stdout: "rank": 0, 2020-09-12T14:07:59.128 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "a", 2020-09-12T14:07:59.128 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:07:59.128 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:07:59.128 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.128 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:07:59.128 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:7115", 2020-09-12T14:07:59.128 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 0 2020-09-12T14:07:59.128 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.129 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.129 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.129 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:7115/0", 2020-09-12T14:07:59.129 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:7115/0", 2020-09-12T14:07:59.129 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 0, 2020-09-12T14:07:59.129 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 0 2020-09-12T14:07:59.129 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.130 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.130 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.130 INFO:tasks.workunit.client.0.smithi027.stdout: "quorum": [ 2020-09-12T14:07:59.130 INFO:tasks.workunit.client.0.smithi027.stdout: 0 2020-09-12T14:07:59.130 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:07:59.130 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap": { 2020-09-12T14:07:59.130 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 40, 2020-09-12T14:07:59.130 INFO:tasks.workunit.client.0.smithi027.stdout: "fsid": "7e90bad3-7973-4a8c-be65-6326ab931fe8", 2020-09-12T14:07:59.131 INFO:tasks.workunit.client.0.smithi027.stdout: "created": "2020-09-12T14:05:44.793120+0000", 2020-09-12T14:07:59.131 INFO:tasks.workunit.client.0.smithi027.stdout: "modified": "2020-09-12T14:07:16.406982+0000", 2020-09-12T14:07:59.131 INFO:tasks.workunit.client.0.smithi027.stdout: "last_up_change": "2020-09-12T14:07:14.683513+0000", 2020-09-12T14:07:59.131 INFO:tasks.workunit.client.0.smithi027.stdout: "last_in_change": "2020-09-12T14:07:16.077966+0000", 2020-09-12T14:07:59.131 INFO:tasks.workunit.client.0.smithi027.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-09-12T14:07:59.131 INFO:tasks.workunit.client.0.smithi027.stdout: "flags_num": 5800384, 2020-09-12T14:07:59.131 INFO:tasks.workunit.client.0.smithi027.stdout: "flags_set": [ 2020-09-12T14:07:59.131 INFO:tasks.workunit.client.0.smithi027.stdout: "nodown", 2020-09-12T14:07:59.132 INFO:tasks.workunit.client.0.smithi027.stdout: "noin", 2020-09-12T14:07:59.132 INFO:tasks.workunit.client.0.smithi027.stdout: "noout", 2020-09-12T14:07:59.132 INFO:tasks.workunit.client.0.smithi027.stdout: "pglog_hardlimit", 2020-09-12T14:07:59.132 INFO:tasks.workunit.client.0.smithi027.stdout: "purged_snapdirs", 2020-09-12T14:07:59.132 INFO:tasks.workunit.client.0.smithi027.stdout: "recovery_deletes", 2020-09-12T14:07:59.132 INFO:tasks.workunit.client.0.smithi027.stdout: "sortbitwise" 2020-09-12T14:07:59.132 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:07:59.133 INFO:tasks.workunit.client.0.smithi027.stdout: "crush_version": 8, 2020-09-12T14:07:59.133 INFO:tasks.workunit.client.0.smithi027.stdout: "full_ratio": 0.99000000953674316, 2020-09-12T14:07:59.133 INFO:tasks.workunit.client.0.smithi027.stdout: "backfillfull_ratio": 0.99000000953674316, 2020-09-12T14:07:59.133 INFO:tasks.workunit.client.0.smithi027.stdout: "nearfull_ratio": 0.85000002384185791, 2020-09-12T14:07:59.133 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_snapshot": "", 2020-09-12T14:07:59.133 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_max": 1, 2020-09-12T14:07:59.133 INFO:tasks.workunit.client.0.smithi027.stdout: "max_osd": 3, 2020-09-12T14:07:59.134 INFO:tasks.workunit.client.0.smithi027.stdout: "require_min_compat_client": "luminous", 2020-09-12T14:07:59.134 INFO:tasks.workunit.client.0.smithi027.stdout: "min_compat_client": "jewel", 2020-09-12T14:07:59.134 INFO:tasks.workunit.client.0.smithi027.stdout: "require_osd_release": "pacific", 2020-09-12T14:07:59.134 INFO:tasks.workunit.client.0.smithi027.stdout: "pools": [ 2020-09-12T14:07:59.134 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.134 INFO:tasks.workunit.client.0.smithi027.stdout: "pool": 1, 2020-09-12T14:07:59.134 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_name": "test", 2020-09-12T14:07:59.134 INFO:tasks.workunit.client.0.smithi027.stdout: "create_time": "2020-09-12T14:06:12.930962+0000", 2020-09-12T14:07:59.135 INFO:tasks.workunit.client.0.smithi027.stdout: "flags": 16389, 2020-09-12T14:07:59.135 INFO:tasks.workunit.client.0.smithi027.stdout: "flags_names": "hashpspool,ec_overwrites,pool_snaps", 2020-09-12T14:07:59.135 INFO:tasks.workunit.client.0.smithi027.stdout: "type": 3, 2020-09-12T14:07:59.135 INFO:tasks.workunit.client.0.smithi027.stdout: "size": 3, 2020-09-12T14:07:59.135 INFO:tasks.workunit.client.0.smithi027.stdout: "min_size": 2, 2020-09-12T14:07:59.135 INFO:tasks.workunit.client.0.smithi027.stdout: "crush_rule": 1, 2020-09-12T14:07:59.135 INFO:tasks.workunit.client.0.smithi027.stdout: "object_hash": 2, 2020-09-12T14:07:59.136 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_autoscale_mode": "off", 2020-09-12T14:07:59.136 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_num": 1, 2020-09-12T14:07:59.136 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_placement_num": 1, 2020-09-12T14:07:59.136 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_placement_num_target": 1, 2020-09-12T14:07:59.136 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_num_target": 1, 2020-09-12T14:07:59.136 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_num_pending": 1, 2020-09-12T14:07:59.136 INFO:tasks.workunit.client.0.smithi027.stdout: "last_pg_merge_meta": { 2020-09-12T14:07:59.136 INFO:tasks.workunit.client.0.smithi027.stdout: "source_pgid": "0.0", 2020-09-12T14:07:59.137 INFO:tasks.workunit.client.0.smithi027.stdout: "ready_epoch": 0, 2020-09-12T14:07:59.137 INFO:tasks.workunit.client.0.smithi027.stdout: "last_epoch_started": 0, 2020-09-12T14:07:59.137 INFO:tasks.workunit.client.0.smithi027.stdout: "last_epoch_clean": 0, 2020-09-12T14:07:59.137 INFO:tasks.workunit.client.0.smithi027.stdout: "source_version": "0'0", 2020-09-12T14:07:59.137 INFO:tasks.workunit.client.0.smithi027.stdout: "target_version": "0'0" 2020-09-12T14:07:59.137 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.137 INFO:tasks.workunit.client.0.smithi027.stdout: "last_change": "30", 2020-09-12T14:07:59.137 INFO:tasks.workunit.client.0.smithi027.stdout: "last_force_op_resend": "0", 2020-09-12T14:07:59.138 INFO:tasks.workunit.client.0.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2020-09-12T14:07:59.138 INFO:tasks.workunit.client.0.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2020-09-12T14:07:59.138 INFO:tasks.workunit.client.0.smithi027.stdout: "auid": 0, 2020-09-12T14:07:59.138 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_mode": "pool", 2020-09-12T14:07:59.138 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_seq": 1, 2020-09-12T14:07:59.138 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_epoch": 30, 2020-09-12T14:07:59.138 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_snaps": [ 2020-09-12T14:07:59.139 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.139 INFO:tasks.workunit.client.0.smithi027.stdout: "snapid": 1, 2020-09-12T14:07:59.139 INFO:tasks.workunit.client.0.smithi027.stdout: "stamp": "2020-09-12T14:06:56.423813+0000", 2020-09-12T14:07:59.139 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "snap1" 2020-09-12T14:07:59.139 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.139 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:07:59.139 INFO:tasks.workunit.client.0.smithi027.stdout: "removed_snaps": "[]", 2020-09-12T14:07:59.140 INFO:tasks.workunit.client.0.smithi027.stdout: "quota_max_bytes": 0, 2020-09-12T14:07:59.140 INFO:tasks.workunit.client.0.smithi027.stdout: "quota_max_objects": 0, 2020-09-12T14:07:59.140 INFO:tasks.workunit.client.0.smithi027.stdout: "tiers": [], 2020-09-12T14:07:59.140 INFO:tasks.workunit.client.0.smithi027.stdout: "tier_of": -1, 2020-09-12T14:07:59.140 INFO:tasks.workunit.client.0.smithi027.stdout: "read_tier": -1, 2020-09-12T14:07:59.140 INFO:tasks.workunit.client.0.smithi027.stdout: "write_tier": -1, 2020-09-12T14:07:59.140 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_mode": "none", 2020-09-12T14:07:59.140 INFO:tasks.workunit.client.0.smithi027.stdout: "target_max_bytes": 0, 2020-09-12T14:07:59.141 INFO:tasks.workunit.client.0.smithi027.stdout: "target_max_objects": 0, 2020-09-12T14:07:59.141 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-09-12T14:07:59.141 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-09-12T14:07:59.141 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2020-09-12T14:07:59.141 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_min_flush_age": 0, 2020-09-12T14:07:59.141 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_min_evict_age": 0, 2020-09-12T14:07:59.141 INFO:tasks.workunit.client.0.smithi027.stdout: "erasure_code_profile": "myprofile", 2020-09-12T14:07:59.141 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_params": { 2020-09-12T14:07:59.142 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "none" 2020-09-12T14:07:59.142 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.142 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_period": 0, 2020-09-12T14:07:59.142 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_count": 0, 2020-09-12T14:07:59.142 INFO:tasks.workunit.client.0.smithi027.stdout: "use_gmt_hitset": true, 2020-09-12T14:07:59.142 INFO:tasks.workunit.client.0.smithi027.stdout: "min_read_recency_for_promote": 0, 2020-09-12T14:07:59.142 INFO:tasks.workunit.client.0.smithi027.stdout: "min_write_recency_for_promote": 0, 2020-09-12T14:07:59.143 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2020-09-12T14:07:59.143 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_search_last_n": 0, 2020-09-12T14:07:59.143 INFO:tasks.workunit.client.0.smithi027.stdout: "grade_table": [], 2020-09-12T14:07:59.143 INFO:tasks.workunit.client.0.smithi027.stdout: "stripe_width": 8192, 2020-09-12T14:07:59.143 INFO:tasks.workunit.client.0.smithi027.stdout: "expected_num_objects": 0, 2020-09-12T14:07:59.143 INFO:tasks.workunit.client.0.smithi027.stdout: "fast_read": false, 2020-09-12T14:07:59.143 INFO:tasks.workunit.client.0.smithi027.stdout: "options": {}, 2020-09-12T14:07:59.143 INFO:tasks.workunit.client.0.smithi027.stdout: "application_metadata": {} 2020-09-12T14:07:59.144 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.144 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:07:59.144 INFO:tasks.workunit.client.0.smithi027.stdout: "osds": [ 2020-09-12T14:07:59.144 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.144 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:07:59.144 INFO:tasks.workunit.client.0.smithi027.stdout: "uuid": "3d802381-3749-43a0-ac4c-e7fa2730ba5f", 2020-09-12T14:07:59.144 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 1, 2020-09-12T14:07:59.145 INFO:tasks.workunit.client.0.smithi027.stdout: "in": 1, 2020-09-12T14:07:59.145 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 1, 2020-09-12T14:07:59.145 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_affinity": 1, 2020-09-12T14:07:59.145 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_begin": 5, 2020-09-12T14:07:59.145 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_end": 29, 2020-09-12T14:07:59.145 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 36, 2020-09-12T14:07:59.145 INFO:tasks.workunit.client.0.smithi027.stdout: "up_thru": 39, 2020-09-12T14:07:59.145 INFO:tasks.workunit.client.0.smithi027.stdout: "down_at": 31, 2020-09-12T14:07:59.146 INFO:tasks.workunit.client.0.smithi027.stdout: "lost_at": 0, 2020-09-12T14:07:59.146 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:07:59.146 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:07:59.146 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.146 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:07:59.146 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6802", 2020-09-12T14:07:59.146 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 60986 2020-09-12T14:07:59.146 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.147 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.147 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:07:59.147 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6803", 2020-09-12T14:07:59.147 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 60986 2020-09-12T14:07:59.147 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.147 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.147 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.148 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addrs": { 2020-09-12T14:07:59.148 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:07:59.148 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.148 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:07:59.148 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6804", 2020-09-12T14:07:59.148 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 60986 2020-09-12T14:07:59.148 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.149 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.149 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:07:59.149 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6805", 2020-09-12T14:07:59.149 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 60986 2020-09-12T14:07:59.149 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.149 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.149 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.149 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addrs": { 2020-09-12T14:07:59.150 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:07:59.150 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.150 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:07:59.150 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6808", 2020-09-12T14:07:59.150 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 60986 2020-09-12T14:07:59.150 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.150 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.151 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:07:59.151 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6809", 2020-09-12T14:07:59.151 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 60986 2020-09-12T14:07:59.151 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.151 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.151 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.151 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addrs": { 2020-09-12T14:07:59.151 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:07:59.152 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.152 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:07:59.152 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6806", 2020-09-12T14:07:59.152 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 60986 2020-09-12T14:07:59.152 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.152 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.152 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:07:59.153 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6807", 2020-09-12T14:07:59.153 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 60986 2020-09-12T14:07:59.153 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.153 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.153 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.153 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:6803/60986", 2020-09-12T14:07:59.153 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addr": "127.0.0.1:6805/60986", 2020-09-12T14:07:59.153 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addr": "127.0.0.1:6809/60986", 2020-09-12T14:07:59.154 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addr": "127.0.0.1:6807/60986", 2020-09-12T14:07:59.154 INFO:tasks.workunit.client.0.smithi027.stdout: "state": [ 2020-09-12T14:07:59.154 INFO:tasks.workunit.client.0.smithi027.stdout: "exists", 2020-09-12T14:07:59.154 INFO:tasks.workunit.client.0.smithi027.stdout: "up" 2020-09-12T14:07:59.154 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.154 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.154 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.154 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:07:59.155 INFO:tasks.workunit.client.0.smithi027.stdout: "uuid": "270594ae-5b9c-4b0c-bd1d-3fddd2a993d0", 2020-09-12T14:07:59.155 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 0, 2020-09-12T14:07:59.155 INFO:tasks.workunit.client.0.smithi027.stdout: "in": 1, 2020-09-12T14:07:59.155 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 1, 2020-09-12T14:07:59.155 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_affinity": 1, 2020-09-12T14:07:59.155 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_begin": 0, 2020-09-12T14:07:59.155 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_end": 0, 2020-09-12T14:07:59.156 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 10, 2020-09-12T14:07:59.156 INFO:tasks.workunit.client.0.smithi027.stdout: "up_thru": 25, 2020-09-12T14:07:59.156 INFO:tasks.workunit.client.0.smithi027.stdout: "down_at": 32, 2020-09-12T14:07:59.156 INFO:tasks.workunit.client.0.smithi027.stdout: "lost_at": 0, 2020-09-12T14:07:59.156 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:07:59.156 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:07:59.156 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.156 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:07:59.157 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6810", 2020-09-12T14:07:59.157 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 55733 2020-09-12T14:07:59.157 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.157 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.157 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:07:59.157 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6811", 2020-09-12T14:07:59.157 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 55733 2020-09-12T14:07:59.157 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.158 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.158 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.158 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addrs": { 2020-09-12T14:07:59.158 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:07:59.158 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.158 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:07:59.158 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6812", 2020-09-12T14:07:59.159 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 55733 2020-09-12T14:07:59.159 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.159 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.159 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:07:59.159 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6813", 2020-09-12T14:07:59.159 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 55733 2020-09-12T14:07:59.159 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.159 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.160 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.160 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addrs": { 2020-09-12T14:07:59.160 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:07:59.160 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.160 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:07:59.160 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6816", 2020-09-12T14:07:59.160 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 55733 2020-09-12T14:07:59.161 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.161 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.161 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:07:59.161 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6817", 2020-09-12T14:07:59.161 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 55733 2020-09-12T14:07:59.161 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.161 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.161 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.162 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addrs": { 2020-09-12T14:07:59.162 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:07:59.162 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.162 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:07:59.162 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6814", 2020-09-12T14:07:59.162 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 55733 2020-09-12T14:07:59.162 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.162 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.163 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:07:59.163 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6815", 2020-09-12T14:07:59.163 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 55733 2020-09-12T14:07:59.163 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.163 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.163 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.163 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:6811/55733", 2020-09-12T14:07:59.164 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addr": "127.0.0.1:6813/55733", 2020-09-12T14:07:59.164 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addr": "127.0.0.1:6817/55733", 2020-09-12T14:07:59.164 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addr": "127.0.0.1:6815/55733", 2020-09-12T14:07:59.164 INFO:tasks.workunit.client.0.smithi027.stdout: "state": [ 2020-09-12T14:07:59.164 INFO:tasks.workunit.client.0.smithi027.stdout: "exists" 2020-09-12T14:07:59.164 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.164 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.164 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.165 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:07:59.165 INFO:tasks.workunit.client.0.smithi027.stdout: "uuid": "ed2b134c-2770-4bad-8557-6c53d9f7b711", 2020-09-12T14:07:59.165 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 1, 2020-09-12T14:07:59.165 INFO:tasks.workunit.client.0.smithi027.stdout: "in": 1, 2020-09-12T14:07:59.165 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 1, 2020-09-12T14:07:59.165 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_affinity": 1, 2020-09-12T14:07:59.165 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_begin": 15, 2020-09-12T14:07:59.165 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_end": 29, 2020-09-12T14:07:59.166 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 37, 2020-09-12T14:07:59.166 INFO:tasks.workunit.client.0.smithi027.stdout: "up_thru": 0, 2020-09-12T14:07:59.166 INFO:tasks.workunit.client.0.smithi027.stdout: "down_at": 33, 2020-09-12T14:07:59.166 INFO:tasks.workunit.client.0.smithi027.stdout: "lost_at": 0, 2020-09-12T14:07:59.166 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:07:59.166 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:07:59.166 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.167 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:07:59.167 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6810", 2020-09-12T14:07:59.167 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 61385 2020-09-12T14:07:59.167 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.167 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.167 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:07:59.167 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6811", 2020-09-12T14:07:59.167 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 61385 2020-09-12T14:07:59.168 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.168 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.168 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.168 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addrs": { 2020-09-12T14:07:59.168 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:07:59.168 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.168 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:07:59.169 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6812", 2020-09-12T14:07:59.169 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 61385 2020-09-12T14:07:59.169 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.169 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.169 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:07:59.169 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6813", 2020-09-12T14:07:59.169 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 61385 2020-09-12T14:07:59.170 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.170 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.170 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.170 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addrs": { 2020-09-12T14:07:59.170 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:07:59.170 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.170 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:07:59.170 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6816", 2020-09-12T14:07:59.171 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 61385 2020-09-12T14:07:59.171 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.171 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.171 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:07:59.171 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6817", 2020-09-12T14:07:59.171 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 61385 2020-09-12T14:07:59.171 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.171 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.172 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.172 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addrs": { 2020-09-12T14:07:59.172 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:07:59.172 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.172 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:07:59.172 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6814", 2020-09-12T14:07:59.172 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 61385 2020-09-12T14:07:59.173 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.173 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.173 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:07:59.173 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6815", 2020-09-12T14:07:59.173 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 61385 2020-09-12T14:07:59.173 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.173 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.173 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.174 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:6811/61385", 2020-09-12T14:07:59.174 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addr": "127.0.0.1:6813/61385", 2020-09-12T14:07:59.174 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addr": "127.0.0.1:6817/61385", 2020-09-12T14:07:59.174 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addr": "127.0.0.1:6815/61385", 2020-09-12T14:07:59.174 INFO:tasks.workunit.client.0.smithi027.stdout: "state": [ 2020-09-12T14:07:59.174 INFO:tasks.workunit.client.0.smithi027.stdout: "exists", 2020-09-12T14:07:59.174 INFO:tasks.workunit.client.0.smithi027.stdout: "up" 2020-09-12T14:07:59.174 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.175 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.175 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:07:59.175 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_xinfo": [ 2020-09-12T14:07:59.175 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.175 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:07:59.175 INFO:tasks.workunit.client.0.smithi027.stdout: "down_stamp": "2020-09-12T14:07:07.559012+0000", 2020-09-12T14:07:59.175 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_probability": 0, 2020-09-12T14:07:59.175 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_interval": 0, 2020-09-12T14:07:59.176 INFO:tasks.workunit.client.0.smithi027.stdout: "features": 4540138292841938943, 2020-09-12T14:07:59.176 INFO:tasks.workunit.client.0.smithi027.stdout: "old_weight": 0, 2020-09-12T14:07:59.176 INFO:tasks.workunit.client.0.smithi027.stdout: "last_purged_snaps_scrub": "2020-09-12T14:05:53.422495+0000", 2020-09-12T14:07:59.176 INFO:tasks.workunit.client.0.smithi027.stdout: "dead_epoch": 0 2020-09-12T14:07:59.176 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.176 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.176 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:07:59.177 INFO:tasks.workunit.client.0.smithi027.stdout: "down_stamp": "2020-09-12T14:07:08.150640+0000", 2020-09-12T14:07:59.177 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_probability": 0, 2020-09-12T14:07:59.177 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_interval": 0, 2020-09-12T14:07:59.177 INFO:tasks.workunit.client.0.smithi027.stdout: "features": 4540138292841938943, 2020-09-12T14:07:59.177 INFO:tasks.workunit.client.0.smithi027.stdout: "old_weight": 0, 2020-09-12T14:07:59.177 INFO:tasks.workunit.client.0.smithi027.stdout: "last_purged_snaps_scrub": "2020-09-12T14:06:01.399666+0000", 2020-09-12T14:07:59.177 INFO:tasks.workunit.client.0.smithi027.stdout: "dead_epoch": 0 2020-09-12T14:07:59.177 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.178 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.178 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:07:59.178 INFO:tasks.workunit.client.0.smithi027.stdout: "down_stamp": "2020-09-12T14:07:08.715577+0000", 2020-09-12T14:07:59.178 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_probability": 0, 2020-09-12T14:07:59.178 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_interval": 0, 2020-09-12T14:07:59.178 INFO:tasks.workunit.client.0.smithi027.stdout: "features": 4540138292841938943, 2020-09-12T14:07:59.178 INFO:tasks.workunit.client.0.smithi027.stdout: "old_weight": 0, 2020-09-12T14:07:59.179 INFO:tasks.workunit.client.0.smithi027.stdout: "last_purged_snaps_scrub": "2020-09-12T14:06:09.531147+0000", 2020-09-12T14:07:59.179 INFO:tasks.workunit.client.0.smithi027.stdout: "dead_epoch": 0 2020-09-12T14:07:59.179 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.179 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:07:59.179 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_upmap": [], 2020-09-12T14:07:59.179 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_upmap_items": [], 2020-09-12T14:07:59.179 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_temp": [], 2020-09-12T14:07:59.179 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_temp": [], 2020-09-12T14:07:59.180 INFO:tasks.workunit.client.0.smithi027.stdout: "blocklist": {}, 2020-09-12T14:07:59.180 INFO:tasks.workunit.client.0.smithi027.stdout: "erasure_code_profiles": { 2020-09-12T14:07:59.180 INFO:tasks.workunit.client.0.smithi027.stdout: "default": { 2020-09-12T14:07:59.180 INFO:tasks.workunit.client.0.smithi027.stdout: "k": "2", 2020-09-12T14:07:59.180 INFO:tasks.workunit.client.0.smithi027.stdout: "m": "2", 2020-09-12T14:07:59.180 INFO:tasks.workunit.client.0.smithi027.stdout: "plugin": "jerasure", 2020-09-12T14:07:59.180 INFO:tasks.workunit.client.0.smithi027.stdout: "technique": "reed_sol_van" 2020-09-12T14:07:59.180 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.181 INFO:tasks.workunit.client.0.smithi027.stdout: "myprofile": { 2020-09-12T14:07:59.181 INFO:tasks.workunit.client.0.smithi027.stdout: "crush-device-class": "", 2020-09-12T14:07:59.181 INFO:tasks.workunit.client.0.smithi027.stdout: "crush-failure-domain": "osd", 2020-09-12T14:07:59.181 INFO:tasks.workunit.client.0.smithi027.stdout: "crush-root": "default", 2020-09-12T14:07:59.181 INFO:tasks.workunit.client.0.smithi027.stdout: "jerasure-per-chunk-alignment": "false", 2020-09-12T14:07:59.181 INFO:tasks.workunit.client.0.smithi027.stdout: "k": "2", 2020-09-12T14:07:59.181 INFO:tasks.workunit.client.0.smithi027.stdout: "m": "1", 2020-09-12T14:07:59.181 INFO:tasks.workunit.client.0.smithi027.stdout: "plugin": "jerasure", 2020-09-12T14:07:59.182 INFO:tasks.workunit.client.0.smithi027.stdout: "technique": "reed_sol_van", 2020-09-12T14:07:59.182 INFO:tasks.workunit.client.0.smithi027.stdout: "w": "8" 2020-09-12T14:07:59.182 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.182 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.182 INFO:tasks.workunit.client.0.smithi027.stdout: "removed_snaps_queue": [], 2020-09-12T14:07:59.182 INFO:tasks.workunit.client.0.smithi027.stdout: "new_removed_snaps": [], 2020-09-12T14:07:59.182 INFO:tasks.workunit.client.0.smithi027.stdout: "new_purged_snaps": [], 2020-09-12T14:07:59.183 INFO:tasks.workunit.client.0.smithi027.stdout: "crush_node_flags": {}, 2020-09-12T14:07:59.183 INFO:tasks.workunit.client.0.smithi027.stdout: "device_class_flags": {} 2020-09-12T14:07:59.183 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.183 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_metadata": [ 2020-09-12T14:07:59.183 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.183 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:07:59.183 INFO:tasks.workunit.client.0.smithi027.stdout: "arch": "x86_64", 2020-09-12T14:07:59.183 INFO:tasks.workunit.client.0.smithi027.stdout: "back_addr": "[v2:127.0.0.1:6804/60986,v1:127.0.0.1:6805/60986]", 2020-09-12T14:07:59.184 INFO:tasks.workunit.client.0.smithi027.stdout: "back_iface": "", 2020-09-12T14:07:59.184 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs": "1", 2020-09-12T14:07:59.184 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_db": "0", 2020-09-12T14:07:59.184 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_wal": "0", 2020-09-12T14:07:59.184 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_single_shared_device": "1", 2020-09-12T14:07:59.184 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_access_mode": "file", 2020-09-12T14:07:59.184 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_block_size": "4096", 2020-09-12T14:07:59.184 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_devices": "sda", 2020-09-12T14:07:59.185 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_driver": "KernelDevice", 2020-09-12T14:07:59.185 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2020-09-12T14:07:59.185 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_rotational": "1", 2020-09-12T14:07:59.185 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_size": "107374182400", 2020-09-12T14:07:59.185 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_support_discard": "0", 2020-09-12T14:07:59.185 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_type": "hdd", 2020-09-12T14:07:59.185 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_release": "pacific", 2020-09-12T14:07:59.186 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version": "ceph version 16.0.0-5485-gf346cc73d15 (f346cc73d15a568d3bec1c99a58953de76f57b85) pacific (dev)", 2020-09-12T14:07:59.186 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version_short": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:07:59.186 INFO:tasks.workunit.client.0.smithi027.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2020-09-12T14:07:59.186 INFO:tasks.workunit.client.0.smithi027.stdout: "default_device_class": "hdd", 2020-09-12T14:07:59.186 INFO:tasks.workunit.client.0.smithi027.stdout: "device_ids": "sda=ST1000NM0033-9ZM173_Z1W4H3N7", 2020-09-12T14:07:59.186 INFO:tasks.workunit.client.0.smithi027.stdout: "device_paths": "sda=/dev/disk/by-path/pci-0000:00:1f.2-ata-1", 2020-09-12T14:07:59.186 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": "sda", 2020-09-12T14:07:59.187 INFO:tasks.workunit.client.0.smithi027.stdout: "distro": "centos", 2020-09-12T14:07:59.187 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_description": "CentOS Linux 8 (Core)", 2020-09-12T14:07:59.187 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_version": "8", 2020-09-12T14:07:59.187 INFO:tasks.workunit.client.0.smithi027.stdout: "front_addr": "[v2:127.0.0.1:6802/60986,v1:127.0.0.1:6803/60986]", 2020-09-12T14:07:59.187 INFO:tasks.workunit.client.0.smithi027.stdout: "front_iface": "", 2020-09-12T14:07:59.187 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/60986,v1:127.0.0.1:6809/60986]", 2020-09-12T14:07:59.187 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/60986,v1:127.0.0.1:6807/60986]", 2020-09-12T14:07:59.187 INFO:tasks.workunit.client.0.smithi027.stdout: "hostname": "smithi027", 2020-09-12T14:07:59.188 INFO:tasks.workunit.client.0.smithi027.stdout: "journal_rotational": "1", 2020-09-12T14:07:59.188 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_description": "#1 SMP Sun Jul 26 03:54:29 UTC 2020", 2020-09-12T14:07:59.188 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_version": "4.18.0-193.14.2.el8_2.x86_64", 2020-09-12T14:07:59.188 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_swap_kb": "0", 2020-09-12T14:07:59.188 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_total_kb": "32659336", 2020-09-12T14:07:59.188 INFO:tasks.workunit.client.0.smithi027.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2020-09-12T14:07:59.188 INFO:tasks.workunit.client.0.smithi027.stdout: "objectstore_numa_unknown_devices": "sda", 2020-09-12T14:07:59.188 INFO:tasks.workunit.client.0.smithi027.stdout: "os": "Linux", 2020-09-12T14:07:59.189 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_data": "td/divergent-priors/0", 2020-09-12T14:07:59.189 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_objectstore": "bluestore", 2020-09-12T14:07:59.189 INFO:tasks.workunit.client.0.smithi027.stdout: "osdspec_affinity": "", 2020-09-12T14:07:59.189 INFO:tasks.workunit.client.0.smithi027.stdout: "rotational": "1" 2020-09-12T14:07:59.189 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.189 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.189 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:07:59.189 INFO:tasks.workunit.client.0.smithi027.stdout: "arch": "x86_64", 2020-09-12T14:07:59.190 INFO:tasks.workunit.client.0.smithi027.stdout: "back_addr": "[v2:127.0.0.1:6812/55733,v1:127.0.0.1:6813/55733]", 2020-09-12T14:07:59.190 INFO:tasks.workunit.client.0.smithi027.stdout: "back_iface": "", 2020-09-12T14:07:59.190 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs": "1", 2020-09-12T14:07:59.190 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_db": "0", 2020-09-12T14:07:59.190 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_wal": "0", 2020-09-12T14:07:59.190 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_single_shared_device": "1", 2020-09-12T14:07:59.190 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_access_mode": "file", 2020-09-12T14:07:59.191 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_block_size": "4096", 2020-09-12T14:07:59.191 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_devices": "sda", 2020-09-12T14:07:59.191 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_driver": "KernelDevice", 2020-09-12T14:07:59.191 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2020-09-12T14:07:59.191 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_rotational": "1", 2020-09-12T14:07:59.191 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_size": "107374182400", 2020-09-12T14:07:59.191 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_support_discard": "0", 2020-09-12T14:07:59.191 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_type": "hdd", 2020-09-12T14:07:59.192 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_release": "pacific", 2020-09-12T14:07:59.192 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version": "ceph version 16.0.0-5485-gf346cc73d15 (f346cc73d15a568d3bec1c99a58953de76f57b85) pacific (dev)", 2020-09-12T14:07:59.192 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version_short": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:07:59.192 INFO:tasks.workunit.client.0.smithi027.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2020-09-12T14:07:59.192 INFO:tasks.workunit.client.0.smithi027.stdout: "default_device_class": "hdd", 2020-09-12T14:07:59.192 INFO:tasks.workunit.client.0.smithi027.stdout: "device_ids": "sda=ST1000NM0033-9ZM173_Z1W4H3N7", 2020-09-12T14:07:59.192 INFO:tasks.workunit.client.0.smithi027.stdout: "device_paths": "sda=/dev/disk/by-path/pci-0000:00:1f.2-ata-1", 2020-09-12T14:07:59.193 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": "sda", 2020-09-12T14:07:59.193 INFO:tasks.workunit.client.0.smithi027.stdout: "distro": "centos", 2020-09-12T14:07:59.193 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_description": "CentOS Linux 8 (Core)", 2020-09-12T14:07:59.193 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_version": "8", 2020-09-12T14:07:59.193 INFO:tasks.workunit.client.0.smithi027.stdout: "front_addr": "[v2:127.0.0.1:6810/55733,v1:127.0.0.1:6811/55733]", 2020-09-12T14:07:59.193 INFO:tasks.workunit.client.0.smithi027.stdout: "front_iface": "", 2020-09-12T14:07:59.193 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/55733,v1:127.0.0.1:6817/55733]", 2020-09-12T14:07:59.194 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/55733,v1:127.0.0.1:6815/55733]", 2020-09-12T14:07:59.194 INFO:tasks.workunit.client.0.smithi027.stdout: "hostname": "smithi027", 2020-09-12T14:07:59.194 INFO:tasks.workunit.client.0.smithi027.stdout: "journal_rotational": "1", 2020-09-12T14:07:59.194 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_description": "#1 SMP Sun Jul 26 03:54:29 UTC 2020", 2020-09-12T14:07:59.194 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_version": "4.18.0-193.14.2.el8_2.x86_64", 2020-09-12T14:07:59.194 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_swap_kb": "0", 2020-09-12T14:07:59.194 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_total_kb": "32659336", 2020-09-12T14:07:59.194 INFO:tasks.workunit.client.0.smithi027.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2020-09-12T14:07:59.195 INFO:tasks.workunit.client.0.smithi027.stdout: "objectstore_numa_unknown_devices": "sda", 2020-09-12T14:07:59.195 INFO:tasks.workunit.client.0.smithi027.stdout: "os": "Linux", 2020-09-12T14:07:59.195 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_data": "td/divergent-priors/1", 2020-09-12T14:07:59.195 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_objectstore": "bluestore", 2020-09-12T14:07:59.195 INFO:tasks.workunit.client.0.smithi027.stdout: "osdspec_affinity": "", 2020-09-12T14:07:59.195 INFO:tasks.workunit.client.0.smithi027.stdout: "rotational": "1" 2020-09-12T14:07:59.195 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.196 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.196 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:07:59.196 INFO:tasks.workunit.client.0.smithi027.stdout: "arch": "x86_64", 2020-09-12T14:07:59.196 INFO:tasks.workunit.client.0.smithi027.stdout: "back_addr": "[v2:127.0.0.1:6812/61385,v1:127.0.0.1:6813/61385]", 2020-09-12T14:07:59.196 INFO:tasks.workunit.client.0.smithi027.stdout: "back_iface": "", 2020-09-12T14:07:59.196 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs": "1", 2020-09-12T14:07:59.196 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_db": "0", 2020-09-12T14:07:59.196 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_wal": "0", 2020-09-12T14:07:59.197 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_single_shared_device": "1", 2020-09-12T14:07:59.197 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_access_mode": "file", 2020-09-12T14:07:59.197 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_block_size": "4096", 2020-09-12T14:07:59.197 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_devices": "sda", 2020-09-12T14:07:59.197 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_driver": "KernelDevice", 2020-09-12T14:07:59.197 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2020-09-12T14:07:59.197 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_rotational": "1", 2020-09-12T14:07:59.198 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_size": "107374182400", 2020-09-12T14:07:59.198 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_support_discard": "0", 2020-09-12T14:07:59.198 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_type": "hdd", 2020-09-12T14:07:59.198 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_release": "pacific", 2020-09-12T14:07:59.198 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version": "ceph version 16.0.0-5485-gf346cc73d15 (f346cc73d15a568d3bec1c99a58953de76f57b85) pacific (dev)", 2020-09-12T14:07:59.198 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version_short": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:07:59.198 INFO:tasks.workunit.client.0.smithi027.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2020-09-12T14:07:59.199 INFO:tasks.workunit.client.0.smithi027.stdout: "default_device_class": "hdd", 2020-09-12T14:07:59.199 INFO:tasks.workunit.client.0.smithi027.stdout: "device_ids": "sda=ST1000NM0033-9ZM173_Z1W4H3N7", 2020-09-12T14:07:59.199 INFO:tasks.workunit.client.0.smithi027.stdout: "device_paths": "sda=/dev/disk/by-path/pci-0000:00:1f.2-ata-1", 2020-09-12T14:07:59.199 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": "sda", 2020-09-12T14:07:59.199 INFO:tasks.workunit.client.0.smithi027.stdout: "distro": "centos", 2020-09-12T14:07:59.199 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_description": "CentOS Linux 8 (Core)", 2020-09-12T14:07:59.199 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_version": "8", 2020-09-12T14:07:59.199 INFO:tasks.workunit.client.0.smithi027.stdout: "front_addr": "[v2:127.0.0.1:6810/61385,v1:127.0.0.1:6811/61385]", 2020-09-12T14:07:59.200 INFO:tasks.workunit.client.0.smithi027.stdout: "front_iface": "", 2020-09-12T14:07:59.200 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/61385,v1:127.0.0.1:6817/61385]", 2020-09-12T14:07:59.200 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/61385,v1:127.0.0.1:6815/61385]", 2020-09-12T14:07:59.200 INFO:tasks.workunit.client.0.smithi027.stdout: "hostname": "smithi027", 2020-09-12T14:07:59.200 INFO:tasks.workunit.client.0.smithi027.stdout: "journal_rotational": "1", 2020-09-12T14:07:59.200 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_description": "#1 SMP Sun Jul 26 03:54:29 UTC 2020", 2020-09-12T14:07:59.200 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_version": "4.18.0-193.14.2.el8_2.x86_64", 2020-09-12T14:07:59.200 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_swap_kb": "0", 2020-09-12T14:07:59.201 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_total_kb": "32659336", 2020-09-12T14:07:59.201 INFO:tasks.workunit.client.0.smithi027.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2020-09-12T14:07:59.201 INFO:tasks.workunit.client.0.smithi027.stdout: "objectstore_numa_unknown_devices": "sda", 2020-09-12T14:07:59.201 INFO:tasks.workunit.client.0.smithi027.stdout: "os": "Linux", 2020-09-12T14:07:59.201 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_data": "td/divergent-priors/2", 2020-09-12T14:07:59.201 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_objectstore": "bluestore", 2020-09-12T14:07:59.201 INFO:tasks.workunit.client.0.smithi027.stdout: "osdspec_affinity": "", 2020-09-12T14:07:59.202 INFO:tasks.workunit.client.0.smithi027.stdout: "rotational": "1" 2020-09-12T14:07:59.202 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.202 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:07:59.202 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap_clean_epochs": { 2020-09-12T14:07:59.202 INFO:tasks.workunit.client.0.smithi027.stdout: "min_last_epoch_clean": 0, 2020-09-12T14:07:59.202 INFO:tasks.workunit.client.0.smithi027.stdout: "last_epoch_clean": { 2020-09-12T14:07:59.202 INFO:tasks.workunit.client.0.smithi027.stdout: "per_pool": [] 2020-09-12T14:07:59.202 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.203 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_epochs": [ 2020-09-12T14:07:59.203 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.203 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:07:59.203 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 36 2020-09-12T14:07:59.203 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.203 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.203 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:07:59.203 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 10 2020-09-12T14:07:59.204 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.204 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.204 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:07:59.204 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 37 2020-09-12T14:07:59.204 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.204 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.204 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.205 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap_first_committed": 1, 2020-09-12T14:07:59.205 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap_last_committed": 40, 2020-09-12T14:07:59.205 INFO:tasks.workunit.client.0.smithi027.stdout: "crushmap": { 2020-09-12T14:07:59.205 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": [ 2020-09-12T14:07:59.205 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.205 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:07:59.205 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd.0", 2020-09-12T14:07:59.206 INFO:tasks.workunit.client.0.smithi027.stdout: "class": "hdd" 2020-09-12T14:07:59.206 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.206 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.206 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:07:59.206 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd.1", 2020-09-12T14:07:59.206 INFO:tasks.workunit.client.0.smithi027.stdout: "class": "hdd" 2020-09-12T14:07:59.206 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.207 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.207 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:07:59.207 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd.2", 2020-09-12T14:07:59.207 INFO:tasks.workunit.client.0.smithi027.stdout: "class": "hdd" 2020-09-12T14:07:59.207 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.207 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:07:59.207 INFO:tasks.workunit.client.0.smithi027.stdout: "types": [ 2020-09-12T14:07:59.207 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.208 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 0, 2020-09-12T14:07:59.208 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd" 2020-09-12T14:07:59.208 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.208 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.208 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 1, 2020-09-12T14:07:59.208 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "host" 2020-09-12T14:07:59.208 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.208 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.209 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 2, 2020-09-12T14:07:59.209 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "chassis" 2020-09-12T14:07:59.209 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.209 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.209 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 3, 2020-09-12T14:07:59.209 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "rack" 2020-09-12T14:07:59.209 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.210 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.210 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 4, 2020-09-12T14:07:59.210 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "row" 2020-09-12T14:07:59.210 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.210 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.210 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 5, 2020-09-12T14:07:59.210 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "pdu" 2020-09-12T14:07:59.211 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.211 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.211 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 6, 2020-09-12T14:07:59.211 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "pod" 2020-09-12T14:07:59.211 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.211 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.211 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 7, 2020-09-12T14:07:59.211 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "room" 2020-09-12T14:07:59.212 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.212 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.212 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 8, 2020-09-12T14:07:59.212 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "datacenter" 2020-09-12T14:07:59.212 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.212 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.212 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 9, 2020-09-12T14:07:59.213 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "zone" 2020-09-12T14:07:59.213 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.213 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.213 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 10, 2020-09-12T14:07:59.213 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "region" 2020-09-12T14:07:59.213 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.213 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.213 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 11, 2020-09-12T14:07:59.214 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "root" 2020-09-12T14:07:59.214 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.214 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:07:59.214 INFO:tasks.workunit.client.0.smithi027.stdout: "buckets": [ 2020-09-12T14:07:59.214 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.214 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -1, 2020-09-12T14:07:59.214 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "default", 2020-09-12T14:07:59.214 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 11, 2020-09-12T14:07:59.215 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "root", 2020-09-12T14:07:59.215 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:07:59.215 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:07:59.215 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:07:59.215 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:07:59.215 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.215 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -3, 2020-09-12T14:07:59.215 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:07:59.216 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:07:59.216 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.216 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.216 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.216 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.216 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -2, 2020-09-12T14:07:59.216 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "default~hdd", 2020-09-12T14:07:59.217 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 11, 2020-09-12T14:07:59.217 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "root", 2020-09-12T14:07:59.217 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:07:59.217 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:07:59.217 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:07:59.217 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:07:59.217 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.217 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -4, 2020-09-12T14:07:59.218 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:07:59.218 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:07:59.218 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.218 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.218 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.218 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.218 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -3, 2020-09-12T14:07:59.218 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "smithi027", 2020-09-12T14:07:59.219 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 1, 2020-09-12T14:07:59.219 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "host", 2020-09-12T14:07:59.219 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:07:59.219 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:07:59.219 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:07:59.219 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:07:59.219 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.219 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:07:59.220 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:07:59.220 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:07:59.220 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.220 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.220 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:07:59.220 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:07:59.220 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 1 2020-09-12T14:07:59.220 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.221 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.221 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:07:59.221 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:07:59.221 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 2 2020-09-12T14:07:59.221 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.221 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.221 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.222 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.222 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -4, 2020-09-12T14:07:59.222 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "smithi027~hdd", 2020-09-12T14:07:59.222 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 1, 2020-09-12T14:07:59.222 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "host", 2020-09-12T14:07:59.222 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:07:59.222 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:07:59.222 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:07:59.223 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:07:59.223 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.223 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:07:59.223 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:07:59.223 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:07:59.223 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.223 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.223 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:07:59.224 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:07:59.224 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 1 2020-09-12T14:07:59.224 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.224 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.224 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:07:59.224 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:07:59.224 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 2 2020-09-12T14:07:59.225 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.225 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.225 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.225 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:07:59.225 INFO:tasks.workunit.client.0.smithi027.stdout: "rules": [ 2020-09-12T14:07:59.225 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.225 INFO:tasks.workunit.client.0.smithi027.stdout: "rule_id": 0, 2020-09-12T14:07:59.225 INFO:tasks.workunit.client.0.smithi027.stdout: "rule_name": "replicated_rule", 2020-09-12T14:07:59.226 INFO:tasks.workunit.client.0.smithi027.stdout: "ruleset": 0, 2020-09-12T14:07:59.226 INFO:tasks.workunit.client.0.smithi027.stdout: "type": 1, 2020-09-12T14:07:59.226 INFO:tasks.workunit.client.0.smithi027.stdout: "min_size": 1, 2020-09-12T14:07:59.226 INFO:tasks.workunit.client.0.smithi027.stdout: "max_size": 10, 2020-09-12T14:07:59.226 INFO:tasks.workunit.client.0.smithi027.stdout: "steps": [ 2020-09-12T14:07:59.226 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.226 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "take", 2020-09-12T14:07:59.226 INFO:tasks.workunit.client.0.smithi027.stdout: "item": -1, 2020-09-12T14:07:59.227 INFO:tasks.workunit.client.0.smithi027.stdout: "item_name": "default" 2020-09-12T14:07:59.227 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.227 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.227 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "choose_firstn", 2020-09-12T14:07:59.227 INFO:tasks.workunit.client.0.smithi027.stdout: "num": 0, 2020-09-12T14:07:59.227 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "osd" 2020-09-12T14:07:59.227 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.227 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.228 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "emit" 2020-09-12T14:07:59.228 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.228 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.228 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.228 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.228 INFO:tasks.workunit.client.0.smithi027.stdout: "rule_id": 1, 2020-09-12T14:07:59.228 INFO:tasks.workunit.client.0.smithi027.stdout: "rule_name": "test", 2020-09-12T14:07:59.229 INFO:tasks.workunit.client.0.smithi027.stdout: "ruleset": 1, 2020-09-12T14:07:59.229 INFO:tasks.workunit.client.0.smithi027.stdout: "type": 3, 2020-09-12T14:07:59.229 INFO:tasks.workunit.client.0.smithi027.stdout: "min_size": 3, 2020-09-12T14:07:59.229 INFO:tasks.workunit.client.0.smithi027.stdout: "max_size": 3, 2020-09-12T14:07:59.229 INFO:tasks.workunit.client.0.smithi027.stdout: "steps": [ 2020-09-12T14:07:59.229 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.229 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "set_chooseleaf_tries", 2020-09-12T14:07:59.229 INFO:tasks.workunit.client.0.smithi027.stdout: "num": 5 2020-09-12T14:07:59.230 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.230 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.230 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "set_choose_tries", 2020-09-12T14:07:59.230 INFO:tasks.workunit.client.0.smithi027.stdout: "num": 100 2020-09-12T14:07:59.230 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.230 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.230 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "take", 2020-09-12T14:07:59.230 INFO:tasks.workunit.client.0.smithi027.stdout: "item": -1, 2020-09-12T14:07:59.231 INFO:tasks.workunit.client.0.smithi027.stdout: "item_name": "default" 2020-09-12T14:07:59.231 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.231 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.231 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "choose_indep", 2020-09-12T14:07:59.231 INFO:tasks.workunit.client.0.smithi027.stdout: "num": 0, 2020-09-12T14:07:59.231 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "osd" 2020-09-12T14:07:59.231 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.231 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.232 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "emit" 2020-09-12T14:07:59.232 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.232 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:07:59.232 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.232 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:07:59.232 INFO:tasks.workunit.client.0.smithi027.stdout: "tunables": { 2020-09-12T14:07:59.232 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_local_tries": 0, 2020-09-12T14:07:59.233 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_local_fallback_tries": 0, 2020-09-12T14:07:59.233 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_total_tries": 50, 2020-09-12T14:07:59.233 INFO:tasks.workunit.client.0.smithi027.stdout: "chooseleaf_descend_once": 1, 2020-09-12T14:07:59.233 INFO:tasks.workunit.client.0.smithi027.stdout: "chooseleaf_vary_r": 1, 2020-09-12T14:07:59.233 INFO:tasks.workunit.client.0.smithi027.stdout: "chooseleaf_stable": 1, 2020-09-12T14:07:59.233 INFO:tasks.workunit.client.0.smithi027.stdout: "straw_calc_version": 1, 2020-09-12T14:07:59.233 INFO:tasks.workunit.client.0.smithi027.stdout: "allowed_bucket_algs": 54, 2020-09-12T14:07:59.233 INFO:tasks.workunit.client.0.smithi027.stdout: "profile": "jewel", 2020-09-12T14:07:59.234 INFO:tasks.workunit.client.0.smithi027.stdout: "optimal_tunables": 1, 2020-09-12T14:07:59.234 INFO:tasks.workunit.client.0.smithi027.stdout: "legacy_tunables": 0, 2020-09-12T14:07:59.234 INFO:tasks.workunit.client.0.smithi027.stdout: "minimum_required_version": "jewel", 2020-09-12T14:07:59.234 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables": 1, 2020-09-12T14:07:59.234 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables2": 1, 2020-09-12T14:07:59.234 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v2_rules": 1, 2020-09-12T14:07:59.234 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables3": 1, 2020-09-12T14:07:59.234 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v3_rules": 0, 2020-09-12T14:07:59.235 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v4_buckets": 1, 2020-09-12T14:07:59.235 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables5": 1, 2020-09-12T14:07:59.235 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v5_rules": 0 2020-09-12T14:07:59.235 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.235 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_args": {} 2020-09-12T14:07:59.235 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.235 INFO:tasks.workunit.client.0.smithi027.stdout: "fsmap": { 2020-09-12T14:07:59.236 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 1, 2020-09-12T14:07:59.236 INFO:tasks.workunit.client.0.smithi027.stdout: "default_fscid": -1, 2020-09-12T14:07:59.236 INFO:tasks.workunit.client.0.smithi027.stdout: "compat": { 2020-09-12T14:07:59.236 INFO:tasks.workunit.client.0.smithi027.stdout: "compat": {}, 2020-09-12T14:07:59.236 INFO:tasks.workunit.client.0.smithi027.stdout: "ro_compat": {}, 2020-09-12T14:07:59.236 INFO:tasks.workunit.client.0.smithi027.stdout: "incompat": { 2020-09-12T14:07:59.236 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_1": "base v0.20", 2020-09-12T14:07:59.236 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_2": "client writeable ranges", 2020-09-12T14:07:59.237 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_3": "default file layouts on dirs", 2020-09-12T14:07:59.237 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_4": "dir inode in separate object", 2020-09-12T14:07:59.237 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_5": "mds uses versioned encoding", 2020-09-12T14:07:59.237 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_6": "dirfrag is stored in omap", 2020-09-12T14:07:59.237 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_8": "no anchor table", 2020-09-12T14:07:59.237 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_9": "file layout v2", 2020-09-12T14:07:59.237 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_10": "snaprealm v2" 2020-09-12T14:07:59.237 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.238 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.238 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_flags": { 2020-09-12T14:07:59.238 INFO:tasks.workunit.client.0.smithi027.stdout: "enable_multiple": false, 2020-09-12T14:07:59.238 INFO:tasks.workunit.client.0.smithi027.stdout: "ever_enabled_multiple": false 2020-09-12T14:07:59.238 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.238 INFO:tasks.workunit.client.0.smithi027.stdout: "standbys": [], 2020-09-12T14:07:59.238 INFO:tasks.workunit.client.0.smithi027.stdout: "filesystems": [] 2020-09-12T14:07:59.238 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.239 INFO:tasks.workunit.client.0.smithi027.stdout: "mdsmap_first_committed": 1, 2020-09-12T14:07:59.239 INFO:tasks.workunit.client.0.smithi027.stdout: "mdsmap_last_committed": 1, 2020-09-12T14:07:59.239 INFO:tasks.workunit.client.0.smithi027.stdout: "auth": { 2020-09-12T14:07:59.239 INFO:tasks.workunit.client.0.smithi027.stdout: "first_committed": 1, 2020-09-12T14:07:59.239 INFO:tasks.workunit.client.0.smithi027.stdout: "last_committed": 5, 2020-09-12T14:07:59.239 INFO:tasks.workunit.client.0.smithi027.stdout: "num_secrets": 3 2020-09-12T14:07:59.239 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.240 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg": 1, 2020-09-12T14:07:59.240 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_active": 1, 2020-09-12T14:07:59.240 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_unknown": 0, 2020-09-12T14:07:59.240 INFO:tasks.workunit.client.0.smithi027.stdout: "num_osd": 3, 2020-09-12T14:07:59.240 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_sum": { 2020-09-12T14:07:59.240 INFO:tasks.workunit.client.0.smithi027.stdout: "stat_sum": { 2020-09-12T14:07:59.240 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes": 48430, 2020-09-12T14:07:59.240 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects": 101, 2020-09-12T14:07:59.241 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_clones": 1, 2020-09-12T14:07:59.241 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_copies": 303, 2020-09-12T14:07:59.241 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing_on_primary": 0, 2020-09-12T14:07:59.241 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing": 0, 2020-09-12T14:07:59.241 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_degraded": 101, 2020-09-12T14:07:59.241 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_misplaced": 0, 2020-09-12T14:07:59.241 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_unfound": 0, 2020-09-12T14:07:59.242 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_dirty": 101, 2020-09-12T14:07:59.242 INFO:tasks.workunit.client.0.smithi027.stdout: "num_whiteouts": 0, 2020-09-12T14:07:59.242 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read": 0, 2020-09-12T14:07:59.242 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read_kb": 0, 2020-09-12T14:07:59.242 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write": 101, 2020-09-12T14:07:59.242 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write_kb": 101, 2020-09-12T14:07:59.242 INFO:tasks.workunit.client.0.smithi027.stdout: "num_scrub_errors": 0, 2020-09-12T14:07:59.242 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shallow_scrub_errors": 0, 2020-09-12T14:07:59.243 INFO:tasks.workunit.client.0.smithi027.stdout: "num_deep_scrub_errors": 0, 2020-09-12T14:07:59.243 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_recovered": 0, 2020-09-12T14:07:59.243 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_recovered": 0, 2020-09-12T14:07:59.243 INFO:tasks.workunit.client.0.smithi027.stdout: "num_keys_recovered": 0, 2020-09-12T14:07:59.243 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_omap": 0, 2020-09-12T14:07:59.243 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_hit_set_archive": 0, 2020-09-12T14:07:59.243 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_hit_set_archive": 0, 2020-09-12T14:07:59.244 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush": 0, 2020-09-12T14:07:59.244 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_kb": 0, 2020-09-12T14:07:59.244 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict": 0, 2020-09-12T14:07:59.244 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_kb": 0, 2020-09-12T14:07:59.244 INFO:tasks.workunit.client.0.smithi027.stdout: "num_promote": 0, 2020-09-12T14:07:59.244 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_high": 0, 2020-09-12T14:07:59.244 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_low": 0, 2020-09-12T14:07:59.244 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_some": 0, 2020-09-12T14:07:59.245 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_full": 0, 2020-09-12T14:07:59.245 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_pinned": 0, 2020-09-12T14:07:59.245 INFO:tasks.workunit.client.0.smithi027.stdout: "num_legacy_snapsets": 0, 2020-09-12T14:07:59.245 INFO:tasks.workunit.client.0.smithi027.stdout: "num_large_omap_objects": 0, 2020-09-12T14:07:59.245 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_manifest": 0, 2020-09-12T14:07:59.245 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_bytes": 0, 2020-09-12T14:07:59.245 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_keys": 0, 2020-09-12T14:07:59.246 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_repaired": 0 2020-09-12T14:07:59.246 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.246 INFO:tasks.workunit.client.0.smithi027.stdout: "store_stats": { 2020-09-12T14:07:59.246 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 0, 2020-09-12T14:07:59.246 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 0, 2020-09-12T14:07:59.246 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:07:59.246 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 0, 2020-09-12T14:07:59.247 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 0, 2020-09-12T14:07:59.247 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:07:59.247 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:07:59.247 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:07:59.247 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 0, 2020-09-12T14:07:59.247 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 0 2020-09-12T14:07:59.247 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.247 INFO:tasks.workunit.client.0.smithi027.stdout: "log_size": 18, 2020-09-12T14:07:59.248 INFO:tasks.workunit.client.0.smithi027.stdout: "ondisk_log_size": 18, 2020-09-12T14:07:59.248 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 3, 2020-09-12T14:07:59.248 INFO:tasks.workunit.client.0.smithi027.stdout: "acting": 3, 2020-09-12T14:07:59.248 INFO:tasks.workunit.client.0.smithi027.stdout: "num_store_stats": 0 2020-09-12T14:07:59.248 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.248 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_sum": { 2020-09-12T14:07:59.248 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 0, 2020-09-12T14:07:59.248 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 0, 2020-09-12T14:07:59.249 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pgs": 3, 2020-09-12T14:07:59.249 INFO:tasks.workunit.client.0.smithi027.stdout: "num_osds": 3, 2020-09-12T14:07:59.249 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_osds": 3, 2020-09-12T14:07:59.249 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_omap_osds": 3, 2020-09-12T14:07:59.249 INFO:tasks.workunit.client.0.smithi027.stdout: "kb": 314572800, 2020-09-12T14:07:59.249 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used": 18404, 2020-09-12T14:07:59.249 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_data": 2116, 2020-09-12T14:07:59.250 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_omap": 1, 2020-09-12T14:07:59.250 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_meta": 16254, 2020-09-12T14:07:59.250 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_avail": 314554396, 2020-09-12T14:07:59.250 INFO:tasks.workunit.client.0.smithi027.stdout: "statfs": { 2020-09-12T14:07:59.250 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 322122547200, 2020-09-12T14:07:59.250 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 322103701504, 2020-09-12T14:07:59.250 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:07:59.250 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 2166784, 2020-09-12T14:07:59.251 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 1523781, 2020-09-12T14:07:59.251 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:07:59.251 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:07:59.251 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:07:59.251 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 1928, 2020-09-12T14:07:59.251 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 16644216 2020-09-12T14:07:59.251 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.251 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_peers": [], 2020-09-12T14:07:59.252 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_trim_queue_len": 0, 2020-09-12T14:07:59.252 INFO:tasks.workunit.client.0.smithi027.stdout: "num_snap_trimming": 0, 2020-09-12T14:07:59.252 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shards_repaired": 0, 2020-09-12T14:07:59.252 INFO:tasks.workunit.client.0.smithi027.stdout: "op_queue_age_hist": { 2020-09-12T14:07:59.252 INFO:tasks.workunit.client.0.smithi027.stdout: "histogram": [], 2020-09-12T14:07:59.252 INFO:tasks.workunit.client.0.smithi027.stdout: "upper_bound": 1 2020-09-12T14:07:59.252 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.253 INFO:tasks.workunit.client.0.smithi027.stdout: "perf_stat": { 2020-09-12T14:07:59.253 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ms": 0, 2020-09-12T14:07:59.253 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ms": 0, 2020-09-12T14:07:59.253 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ns": 0, 2020-09-12T14:07:59.253 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ns": 0 2020-09-12T14:07:59.253 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.253 INFO:tasks.workunit.client.0.smithi027.stdout: "alerts": [], 2020-09-12T14:07:59.253 INFO:tasks.workunit.client.0.smithi027.stdout: "network_ping_times": [] 2020-09-12T14:07:59.254 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.254 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_sum_by_class": { 2020-09-12T14:07:59.254 INFO:tasks.workunit.client.0.smithi027.stdout: "hdd": { 2020-09-12T14:07:59.254 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 0, 2020-09-12T14:07:59.254 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 0, 2020-09-12T14:07:59.254 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pgs": 3, 2020-09-12T14:07:59.254 INFO:tasks.workunit.client.0.smithi027.stdout: "num_osds": 3, 2020-09-12T14:07:59.255 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_osds": 3, 2020-09-12T14:07:59.255 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_omap_osds": 3, 2020-09-12T14:07:59.255 INFO:tasks.workunit.client.0.smithi027.stdout: "kb": 314572800, 2020-09-12T14:07:59.255 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used": 18404, 2020-09-12T14:07:59.255 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_data": 2116, 2020-09-12T14:07:59.255 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_omap": 1, 2020-09-12T14:07:59.255 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_meta": 16254, 2020-09-12T14:07:59.255 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_avail": 314554396, 2020-09-12T14:07:59.256 INFO:tasks.workunit.client.0.smithi027.stdout: "statfs": { 2020-09-12T14:07:59.256 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 322122547200, 2020-09-12T14:07:59.256 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 322103701504, 2020-09-12T14:07:59.256 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:07:59.256 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 2166784, 2020-09-12T14:07:59.256 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 1523781, 2020-09-12T14:07:59.256 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:07:59.256 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:07:59.257 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:07:59.257 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 1928, 2020-09-12T14:07:59.257 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 16644216 2020-09-12T14:07:59.257 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.257 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_peers": [], 2020-09-12T14:07:59.257 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_trim_queue_len": 0, 2020-09-12T14:07:59.257 INFO:tasks.workunit.client.0.smithi027.stdout: "num_snap_trimming": 0, 2020-09-12T14:07:59.257 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shards_repaired": 0, 2020-09-12T14:07:59.258 INFO:tasks.workunit.client.0.smithi027.stdout: "op_queue_age_hist": { 2020-09-12T14:07:59.258 INFO:tasks.workunit.client.0.smithi027.stdout: "histogram": [], 2020-09-12T14:07:59.258 INFO:tasks.workunit.client.0.smithi027.stdout: "upper_bound": 1 2020-09-12T14:07:59.258 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.258 INFO:tasks.workunit.client.0.smithi027.stdout: "perf_stat": { 2020-09-12T14:07:59.258 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ms": 0, 2020-09-12T14:07:59.258 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ms": 0, 2020-09-12T14:07:59.259 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ns": 0, 2020-09-12T14:07:59.259 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ns": 0 2020-09-12T14:07:59.259 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.259 INFO:tasks.workunit.client.0.smithi027.stdout: "alerts": [], 2020-09-12T14:07:59.259 INFO:tasks.workunit.client.0.smithi027.stdout: "network_ping_times": [] 2020-09-12T14:07:59.259 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.259 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.259 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_stats": [ 2020-09-12T14:07:59.260 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.260 INFO:tasks.workunit.client.0.smithi027.stdout: "poolid": 1, 2020-09-12T14:07:59.260 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg": 1, 2020-09-12T14:07:59.260 INFO:tasks.workunit.client.0.smithi027.stdout: "stat_sum": { 2020-09-12T14:07:59.260 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes": 48430, 2020-09-12T14:07:59.260 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects": 101, 2020-09-12T14:07:59.260 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_clones": 1, 2020-09-12T14:07:59.260 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_copies": 303, 2020-09-12T14:07:59.261 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing_on_primary": 0, 2020-09-12T14:07:59.261 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing": 0, 2020-09-12T14:07:59.261 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_degraded": 101, 2020-09-12T14:07:59.261 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_misplaced": 0, 2020-09-12T14:07:59.261 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_unfound": 0, 2020-09-12T14:07:59.261 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_dirty": 101, 2020-09-12T14:07:59.261 INFO:tasks.workunit.client.0.smithi027.stdout: "num_whiteouts": 0, 2020-09-12T14:07:59.262 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read": 0, 2020-09-12T14:07:59.262 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read_kb": 0, 2020-09-12T14:07:59.262 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write": 101, 2020-09-12T14:07:59.262 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write_kb": 101, 2020-09-12T14:07:59.262 INFO:tasks.workunit.client.0.smithi027.stdout: "num_scrub_errors": 0, 2020-09-12T14:07:59.262 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shallow_scrub_errors": 0, 2020-09-12T14:07:59.262 INFO:tasks.workunit.client.0.smithi027.stdout: "num_deep_scrub_errors": 0, 2020-09-12T14:07:59.262 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_recovered": 0, 2020-09-12T14:07:59.263 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_recovered": 0, 2020-09-12T14:07:59.263 INFO:tasks.workunit.client.0.smithi027.stdout: "num_keys_recovered": 0, 2020-09-12T14:07:59.263 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_omap": 0, 2020-09-12T14:07:59.263 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_hit_set_archive": 0, 2020-09-12T14:07:59.263 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_hit_set_archive": 0, 2020-09-12T14:07:59.263 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush": 0, 2020-09-12T14:07:59.263 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_kb": 0, 2020-09-12T14:07:59.263 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict": 0, 2020-09-12T14:07:59.264 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_kb": 0, 2020-09-12T14:07:59.264 INFO:tasks.workunit.client.0.smithi027.stdout: "num_promote": 0, 2020-09-12T14:07:59.264 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_high": 0, 2020-09-12T14:07:59.264 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_low": 0, 2020-09-12T14:07:59.264 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_some": 0, 2020-09-12T14:07:59.264 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_full": 0, 2020-09-12T14:07:59.264 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_pinned": 0, 2020-09-12T14:07:59.265 INFO:tasks.workunit.client.0.smithi027.stdout: "num_legacy_snapsets": 0, 2020-09-12T14:07:59.265 INFO:tasks.workunit.client.0.smithi027.stdout: "num_large_omap_objects": 0, 2020-09-12T14:07:59.265 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_manifest": 0, 2020-09-12T14:07:59.265 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_bytes": 0, 2020-09-12T14:07:59.265 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_keys": 0, 2020-09-12T14:07:59.265 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_repaired": 0 2020-09-12T14:07:59.265 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.265 INFO:tasks.workunit.client.0.smithi027.stdout: "store_stats": { 2020-09-12T14:07:59.266 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 0, 2020-09-12T14:07:59.266 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 0, 2020-09-12T14:07:59.266 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:07:59.266 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 1241088, 2020-09-12T14:07:59.266 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 1241088, 2020-09-12T14:07:59.266 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:07:59.266 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:07:59.267 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:07:59.267 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 0, 2020-09-12T14:07:59.267 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 0 2020-09-12T14:07:59.267 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.267 INFO:tasks.workunit.client.0.smithi027.stdout: "log_size": 18, 2020-09-12T14:07:59.267 INFO:tasks.workunit.client.0.smithi027.stdout: "ondisk_log_size": 18, 2020-09-12T14:07:59.267 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 3, 2020-09-12T14:07:59.268 INFO:tasks.workunit.client.0.smithi027.stdout: "acting": 3, 2020-09-12T14:07:59.268 INFO:tasks.workunit.client.0.smithi027.stdout: "num_store_stats": 3 2020-09-12T14:07:59.268 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.268 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:07:59.268 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_stats": [ 2020-09-12T14:07:59.268 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.268 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:07:59.268 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 154618822667 2020-09-12T14:07:59.269 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.269 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.269 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:07:59.269 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 42949672982 2020-09-12T14:07:59.269 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.269 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.269 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:07:59.269 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 158913789963 2020-09-12T14:07:59.270 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.270 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:07:59.270 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_by_state": [ 2020-09-12T14:07:59.270 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.270 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "active+undersized+degraded", 2020-09-12T14:07:59.270 INFO:tasks.workunit.client.0.smithi027.stdout: "num": 1 2020-09-12T14:07:59.270 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.271 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:07:59.271 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_by_osd": [ 2020-09-12T14:07:59.271 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.271 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:07:59.271 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 0, 2020-09-12T14:07:59.271 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 1, 2020-09-12T14:07:59.271 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T14:07:59.271 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.272 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.272 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2147483647, 2020-09-12T14:07:59.272 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 0, 2020-09-12T14:07:59.272 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 1, 2020-09-12T14:07:59.272 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T14:07:59.272 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.272 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.272 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:07:59.273 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 0, 2020-09-12T14:07:59.273 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 0, 2020-09-12T14:07:59.273 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T14:07:59.273 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.273 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.273 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:07:59.273 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 1, 2020-09-12T14:07:59.273 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 1, 2020-09-12T14:07:59.274 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T14:07:59.274 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.274 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:07:59.274 INFO:tasks.workunit.client.0.smithi027.stdout: "purged_snaps": [ 2020-09-12T14:07:59.274 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:07:59.274 INFO:tasks.workunit.client.0.smithi027.stdout: "pool": 1, 2020-09-12T14:07:59.274 INFO:tasks.workunit.client.0.smithi027.stdout: "purged_snaps": {} 2020-09-12T14:07:59.275 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.275 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:07:59.275 INFO:tasks.workunit.client.0.smithi027.stdout: "servicemap": { 2020-09-12T14:07:59.275 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 5, 2020-09-12T14:07:59.275 INFO:tasks.workunit.client.0.smithi027.stdout: "modified": "2020-09-12T14:07:11.322124+0000", 2020-09-12T14:07:59.275 INFO:tasks.workunit.client.0.smithi027.stdout: "services": {} 2020-09-12T14:07:59.275 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:07:59.275 INFO:tasks.workunit.client.0.smithi027.stdout: "paxos": { 2020-09-12T14:07:59.276 INFO:tasks.workunit.client.0.smithi027.stdout: "first_committed": 1, 2020-09-12T14:07:59.276 INFO:tasks.workunit.client.0.smithi027.stdout: "last_committed": 392, 2020-09-12T14:07:59.276 INFO:tasks.workunit.client.0.smithi027.stdout: "last_pn": 0, 2020-09-12T14:07:59.276 INFO:tasks.workunit.client.0.smithi027.stdout: "accepted_pn": 0 2020-09-12T14:07:59.276 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:07:59.276 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:07:59.278 INFO:tasks.workunit.client.0.smithi027.stderr:report 3363646705 2020-09-12T14:07:59.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1604: wait_for_clean: return 1 2020-09-12T14:07:59.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:330: TEST_divergent_ec: for i in $non_divergent 2020-09-12T14:07:59.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:332: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd.0 2020-09-12T14:07:59.278 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:07:59.278 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:07:59.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:07:59.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:07:59.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:07:59.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:07:59.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:333: TEST_divergent_ec: _objectstore_tool_nodown td/divergent-priors 0 --op log --pgid 1.0 2020-09-12T14:07:59.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1195: _objectstore_tool_nodown: local dir=td/divergent-priors 2020-09-12T14:07:59.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: _objectstore_tool_nodown: shift 2020-09-12T14:07:59.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: _objectstore_tool_nodown: local id=0 2020-09-12T14:07:59.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1198: _objectstore_tool_nodown: shift 2020-09-12T14:07:59.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: _objectstore_tool_nodown: local osd_data=td/divergent-priors/0 2020-09-12T14:07:59.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1202: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/0 --op log --pgid 1.0 2020-09-12T14:08:00.106 INFO:tasks.workunit.client.0.smithi027.stderr:PG '1.0' not found 2020-09-12T14:08:00.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1203: _objectstore_tool_nodown: return 1 2020-09-12T14:08:00.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:334: TEST_divergent_ec: activate_osd td/divergent-priors 0 2020-09-12T14:08:00.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/divergent-priors 2020-09-12T14:08:00.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T14:08:00.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=0 2020-09-12T14:08:00.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T14:08:00.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/divergent-priors/0 2020-09-12T14:08:00.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T14:08:00.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:08:00.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:08:00.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:08:00.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2020-09-12T14:08:00.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2020-09-12T14:08:00.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T14:08:00.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T14:08:00.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T14:08:00.383 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T14:08:00.383 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:08:00.384 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:08:00.384 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:08:00.384 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:08:00.384 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:08:00.384 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:08:00.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:08:00.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:08:00.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T14:08:00.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T14:08:00.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:08:00.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:08:00.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:08:00.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T14:08:00.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T14:08:00.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/divergent-priors/0 2020-09-12T14:08:00.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.0 2020-09-12T14:08:00.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 0 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:08:00.386 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/divergent-priors/0/whoami 2020-09-12T14:08:00.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 0 = 0 ']' 2020-09-12T14:08:00.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T14:08:00.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T14:08:00.387 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.0 2020-09-12T14:08:00.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T14:08:00.413 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:08:00.413+0000 7f9880c5bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:08:00.415 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:08:00.415+0000 7f9880c5bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:08:00.417 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:08:00.416+0000 7f9880c5bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:08:00.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: wait_for_osd up 0 2020-09-12T14:08:00.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:08:00.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T14:08:00.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:08:00.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:08:00.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:08:00.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:08:00.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:08:00.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:08:00.812 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:08:00.935 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:08:00.934+0000 7f9880c5bec0 -1 Falling back to public interface 2020-09-12T14:08:01.231 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 36 up_thru 39 down_at 31 last_clean_interval [5,29) [v2:127.0.0.1:6802/60986,v1:127.0.0.1:6803/60986] [v2:127.0.0.1:6804/60986,v1:127.0.0.1:6805/60986] exists,up 3d802381-3749-43a0-ac4c-e7fa2730ba5f 2020-09-12T14:08:01.231 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:08:01.231 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:08:01.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:08:01.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:330: TEST_divergent_ec: for i in $non_divergent 2020-09-12T14:08:01.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:332: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd.2 2020-09-12T14:08:01.232 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:08:01.232 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:08:01.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:08:01.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:08:01.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:08:01.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:08:01.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:333: TEST_divergent_ec: _objectstore_tool_nodown td/divergent-priors 2 --op log --pgid 1.0 2020-09-12T14:08:01.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1195: _objectstore_tool_nodown: local dir=td/divergent-priors 2020-09-12T14:08:01.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: _objectstore_tool_nodown: shift 2020-09-12T14:08:01.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: _objectstore_tool_nodown: local id=2 2020-09-12T14:08:01.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1198: _objectstore_tool_nodown: shift 2020-09-12T14:08:01.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: _objectstore_tool_nodown: local osd_data=td/divergent-priors/2 2020-09-12T14:08:01.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1202: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/2 --op log --pgid 1.0 2020-09-12T14:08:02.035 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:08:02.034+0000 7f9880c5bec0 -1 osd.0 40 log_to_monitors {default=true} 2020-09-12T14:08:02.729 INFO:tasks.workunit.client.0.smithi027.stderr:PG '1.0' not found 2020-09-12T14:08:03.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1203: _objectstore_tool_nodown: return 1 2020-09-12T14:08:03.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:334: TEST_divergent_ec: activate_osd td/divergent-priors 2 2020-09-12T14:08:03.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/divergent-priors 2020-09-12T14:08:03.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T14:08:03.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=2 2020-09-12T14:08:03.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T14:08:03.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/divergent-priors/2 2020-09-12T14:08:03.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T14:08:03.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:08:03.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:08:03.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:08:03.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2020-09-12T14:08:03.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2020-09-12T14:08:03.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T14:08:03.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T14:08:03.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T14:08:03.004 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T14:08:03.004 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:08:03.004 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:08:03.004 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:08:03.004 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:08:03.004 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:08:03.005 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:08:03.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:08:03.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:08:03.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T14:08:03.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T14:08:03.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:08:03.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:08:03.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:08:03.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T14:08:03.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T14:08:03.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/divergent-priors/2 2020-09-12T14:08:03.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.2 2020-09-12T14:08:03.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 2 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:08:03.008 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/divergent-priors/2/whoami 2020-09-12T14:08:03.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 2 = 2 ']' 2020-09-12T14:08:03.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T14:08:03.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T14:08:03.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T14:08:03.008 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.2 2020-09-12T14:08:03.035 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:08:03.035+0000 7fa22f50cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:08:03.037 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:08:03.037+0000 7fa22f50cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:08:03.038 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:08:03.038+0000 7fa22f50cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:08:03.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: wait_for_osd up 2 2020-09-12T14:08:03.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:08:03.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T14:08:03.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:08:03.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:08:03.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:08:03.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:08:03.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:08:03.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:08:03.486 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:08:03.557 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:08:03.557+0000 7fa22f50cec0 -1 Falling back to public interface 2020-09-12T14:08:03.901 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up in weight 1 up_from 37 up_thru 0 down_at 33 last_clean_interval [15,29) [v2:127.0.0.1:6810/61385,v1:127.0.0.1:6811/61385] [v2:127.0.0.1:6812/61385,v1:127.0.0.1:6813/61385] exists,up ed2b134c-2770-4bad-8557-6c53d9f7b711 2020-09-12T14:08:03.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:08:03.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:08:03.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:08:03.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:336: TEST_divergent_ec: _objectstore_tool_nodown td/divergent-priors 1 --op log --pgid 1.0 2020-09-12T14:08:03.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1195: _objectstore_tool_nodown: local dir=td/divergent-priors 2020-09-12T14:08:03.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: _objectstore_tool_nodown: shift 2020-09-12T14:08:03.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: _objectstore_tool_nodown: local id=1 2020-09-12T14:08:03.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1198: _objectstore_tool_nodown: shift 2020-09-12T14:08:03.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2020-09-12T14:08:03.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1202: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op log --pgid 1.0 2020-09-12T14:08:04.559 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:08:04.558+0000 7fa22f50cec0 -1 osd.2 40 log_to_monitors {default=true} 2020-09-12T14:08:05.702 INFO:tasks.workunit.client.0.smithi027.stderr:PG '1.0' not found 2020-09-12T14:08:05.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1203: _objectstore_tool_nodown: return 1 2020-09-12T14:08:05.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:338: TEST_divergent_ec: WAIT_FOR_CLEAN_TIMEOUT=20 2020-09-12T14:08:05.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:338: TEST_divergent_ec: wait_for_clean 2020-09-12T14:08:05.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:08:05.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:08:05.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:08:05.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:08:05.971 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 20 .1 2020-09-12T14:08:05.971 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:08:05.971 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:08:05.971 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:08:05.971 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:08:05.972 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:08:06.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:08:06.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:08:06.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:08:06.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:08:06.041 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:08:06.457 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:08:06.457 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:08:06.457 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T14:08:06.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:08:06.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:08:06.458 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:08:06.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=180388626435 2020-09-12T14:08:06.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 180388626435 2020-09-12T14:08:06.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-180388626435' 2020-09-12T14:08:06.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:08:06.610 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:08:06.716 INFO:tasks.workunit.client.0.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-09-12T14:08:06.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq= 2020-09-12T14:08:06.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z '' 2020-09-12T14:08:06.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2075: flush_pg_stats: continue 2020-09-12T14:08:06.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:08:06.719 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:08:06.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=193273528323 2020-09-12T14:08:06.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 193273528323 2020-09-12T14:08:06.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-180388626435 2-193273528323' 2020-09-12T14:08:06.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:08:06.866 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-180388626435 2020-09-12T14:08:06.866 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:08:06.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:08:06.867 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-180388626435 2020-09-12T14:08:06.867 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:08:06.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=180388626435 2020-09-12T14:08:06.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 180388626435' 2020-09-12T14:08:06.868 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 180388626435 2020-09-12T14:08:06.868 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:08:07.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 1 -lt 180388626435 2020-09-12T14:08:07.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:08:08.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:08:08.280 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:08:08.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 180388626436 -lt 180388626435 2020-09-12T14:08:08.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:08:08.698 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-193273528323 2020-09-12T14:08:08.698 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:08:08.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:08:08.699 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-193273528323 2020-09-12T14:08:08.699 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:08:08.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=193273528323 2020-09-12T14:08:08.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 193273528323' 2020-09-12T14:08:08.700 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 193273528323 2020-09-12T14:08:08.700 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:08:09.116 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 193273528323 -lt 193273528323 2020-09-12T14:08:09.116 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:08:09.117 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:08:09.117 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:08:09.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 1 == 0 2020-09-12T14:08:09.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:08:09.674 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:08:09.674 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:08:09.675 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:08:09.675 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:08:09.675 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:08:09.675 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:08:10.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:08:10.071 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:08:10.072 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:08:10.072 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:08:10.604 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:08:10.604 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' -1 2020-09-12T14:08:10.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1598: wait_for_clean: loop=0 2020-09-12T14:08:10.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1599: wait_for_clean: num_active_clean=0 2020-09-12T14:08:10.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:08:10.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:08:10.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:08:10.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:08:10.706 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:08:10.706 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:08:10.706 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:08:10.706 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:08:10.707 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:08:10.707 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:08:11.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:08:11.097 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:08:11.098 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:08:11.098 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:08:11.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:08:11.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:08:11.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:08:11.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:08:11.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:08:11.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:08:11.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:08:11.637 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:08:11.638 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:08:12.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:08:12.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:08:12.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 1 >= 8 )) 2020-09-12T14:08:12.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:08:12.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.2 2020-09-12T14:08:12.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:08:12.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:08:12.385 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:08:12.386 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:08:12.386 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:08:12.386 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:08:12.386 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:08:12.386 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:08:12.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:08:12.789 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:08:12.789 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:08:12.789 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:08:13.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:08:13.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:08:13.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:08:13.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:08:13.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:08:13.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:08:13.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:08:13.329 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:08:13.329 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:08:13.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:08:13.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:08:13.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 2 >= 8 )) 2020-09-12T14:08:13.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:08:13.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.4 2020-09-12T14:08:14.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:08:14.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:08:14.268 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:08:14.268 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:08:14.268 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:08:14.269 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:08:14.269 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:08:14.269 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:08:14.667 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:08:14.668 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:08:14.668 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:08:14.668 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:08:15.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:08:15.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:08:15.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:08:15.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:08:15.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:08:15.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:08:15.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:08:15.208 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:08:15.209 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:08:15.765 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:08:15.766 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:08:15.766 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 3 >= 8 )) 2020-09-12T14:08:15.766 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:08:15.766 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.8 2020-09-12T14:08:16.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:08:16.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:08:16.567 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:08:16.567 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:08:16.567 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:08:16.568 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:08:16.568 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:08:16.568 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:08:16.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:08:16.977 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:08:16.977 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:08:16.978 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:08:17.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:08:17.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:08:17.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:08:17.510 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:08:17.510 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:08:17.510 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:08:17.510 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:08:17.511 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:08:17.511 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:08:18.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:08:18.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:08:18.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 4 >= 8 )) 2020-09-12T14:08:18.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:08:18.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 1.6 2020-09-12T14:08:19.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:08:19.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:08:19.658 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:08:19.658 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:08:19.659 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:08:19.659 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:08:19.660 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:08:19.660 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:08:20.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:08:20.061 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:08:20.062 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:08:20.062 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:08:20.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:08:20.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:08:20.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:08:20.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:08:20.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:08:20.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:08:20.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:08:20.687 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:08:20.688 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:08:21.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:08:21.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:08:21.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 5 >= 8 )) 2020-09-12T14:08:21.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:08:21.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 3.2 2020-09-12T14:08:24.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:08:24.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:08:24.459 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:08:24.459 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:08:24.460 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:08:24.460 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:08:24.460 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:08:24.460 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:08:24.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:08:24.868 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:08:24.868 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:08:24.869 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:08:25.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:08:25.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:08:25.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:08:25.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:08:25.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:08:25.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:08:25.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:08:25.402 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:08:25.402 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:08:25.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:08:25.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:08:25.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 6 >= 8 )) 2020-09-12T14:08:25.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:08:25.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 6.4 2020-09-12T14:08:32.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:08:32.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:08:32.350 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:08:32.350 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:08:32.350 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:08:32.350 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:08:32.351 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:08:32.351 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:08:32.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:08:32.758 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:08:32.758 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:08:32.758 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:08:33.291 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:08:33.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:08:33.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:08:33.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:08:33.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:08:33.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:08:33.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:08:33.293 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:08:33.293 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:08:33.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:08:33.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:08:33.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 7 >= 8 )) 2020-09-12T14:08:33.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:08:33.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 7.3 2020-09-12T14:08:41.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:08:41.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:08:41.148 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:08:41.149 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:08:41.149 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:08:41.149 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:08:41.149 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:08:41.149 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:08:41.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:08:41.563 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:08:41.564 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:08:41.564 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:08:42.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:08:42.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:08:42.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:08:42.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:08:42.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:08:42.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:08:42.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:08:42.099 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:08:42.100 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:08:42.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=null 2020-09-12T14:08:42.644 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test null '!=' null 2020-09-12T14:08:42.644 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: wait_for_clean: (( 8 >= 8 )) 2020-09-12T14:08:42.644 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: wait_for_clean: ceph report 2020-09-12T14:08:43.166 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:08:43.166 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_fingerprint": "d3487fa8-4c06-46da-bbe3-8d05ca5a9357", 2020-09-12T14:08:43.167 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:08:43.167 INFO:tasks.workunit.client.0.smithi027.stdout: "commit": "f346cc73d15a568d3bec1c99a58953de76f57b85", 2020-09-12T14:08:43.167 INFO:tasks.workunit.client.0.smithi027.stdout: "timestamp": "2020-09-12T14:08:43.165174+0000", 2020-09-12T14:08:43.167 INFO:tasks.workunit.client.0.smithi027.stdout: "tag": "", 2020-09-12T14:08:43.167 INFO:tasks.workunit.client.0.smithi027.stdout: "health": { 2020-09-12T14:08:43.167 INFO:tasks.workunit.client.0.smithi027.stdout: "status": "HEALTH_WARN", 2020-09-12T14:08:43.167 INFO:tasks.workunit.client.0.smithi027.stdout: "checks": { 2020-09-12T14:08:43.168 INFO:tasks.workunit.client.0.smithi027.stdout: "OSDMAP_FLAGS": { 2020-09-12T14:08:43.168 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:08:43.168 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:08:43.168 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "nodown,noout,noin flag(s) set", 2020-09-12T14:08:43.168 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 17 2020-09-12T14:08:43.168 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.168 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [], 2020-09-12T14:08:43.169 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:08:43.169 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.169 INFO:tasks.workunit.client.0.smithi027.stdout: "OSD_DOWN": { 2020-09-12T14:08:43.169 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:08:43.169 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:08:43.169 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "1 osds down", 2020-09-12T14:08:43.169 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 1 2020-09-12T14:08:43.169 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.170 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [ 2020-09-12T14:08:43.170 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.170 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "osd.1 (root=default,host=smithi027) is down" 2020-09-12T14:08:43.170 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.170 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:08:43.170 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:08:43.170 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.171 INFO:tasks.workunit.client.0.smithi027.stdout: "PG_DEGRADED": { 2020-09-12T14:08:43.171 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:08:43.171 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:08:43.171 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "Degraded data redundancy: 101/303 objects degraded (33.333%), 1 pg degraded", 2020-09-12T14:08:43.171 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 1 2020-09-12T14:08:43.171 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.171 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [ 2020-09-12T14:08:43.171 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.172 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [2147483647,0,2]" 2020-09-12T14:08:43.172 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.172 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:08:43.172 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:08:43.172 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.172 INFO:tasks.workunit.client.0.smithi027.stdout: "POOL_APP_NOT_ENABLED": { 2020-09-12T14:08:43.172 INFO:tasks.workunit.client.0.smithi027.stdout: "severity": "HEALTH_WARN", 2020-09-12T14:08:43.172 INFO:tasks.workunit.client.0.smithi027.stdout: "summary": { 2020-09-12T14:08:43.173 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "1 pool(s) do not have an application enabled", 2020-09-12T14:08:43.173 INFO:tasks.workunit.client.0.smithi027.stdout: "count": 1 2020-09-12T14:08:43.173 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.173 INFO:tasks.workunit.client.0.smithi027.stdout: "detail": [ 2020-09-12T14:08:43.173 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.173 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "application not enabled on pool 'test'" 2020-09-12T14:08:43.173 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.173 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.174 INFO:tasks.workunit.client.0.smithi027.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2020-09-12T14:08:43.174 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.174 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:08:43.174 INFO:tasks.workunit.client.0.smithi027.stdout: "muted": false 2020-09-12T14:08:43.174 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.174 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.174 INFO:tasks.workunit.client.0.smithi027.stdout: "mutes": [] 2020-09-12T14:08:43.175 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.175 INFO:tasks.workunit.client.0.smithi027.stdout: "monmap_first_committed": 1, 2020-09-12T14:08:43.175 INFO:tasks.workunit.client.0.smithi027.stdout: "monmap_last_committed": 1, 2020-09-12T14:08:43.175 INFO:tasks.workunit.client.0.smithi027.stdout: "monmap": { 2020-09-12T14:08:43.175 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 1, 2020-09-12T14:08:43.175 INFO:tasks.workunit.client.0.smithi027.stdout: "fsid": "7e90bad3-7973-4a8c-be65-6326ab931fe8", 2020-09-12T14:08:43.175 INFO:tasks.workunit.client.0.smithi027.stdout: "modified": "2020-09-12T14:05:44.022891Z", 2020-09-12T14:08:43.175 INFO:tasks.workunit.client.0.smithi027.stdout: "created": "2020-09-12T14:05:44.022891Z", 2020-09-12T14:08:43.176 INFO:tasks.workunit.client.0.smithi027.stdout: "min_mon_release": 16, 2020-09-12T14:08:43.176 INFO:tasks.workunit.client.0.smithi027.stdout: "min_mon_release_name": "pacific", 2020-09-12T14:08:43.176 INFO:tasks.workunit.client.0.smithi027.stdout: "features": { 2020-09-12T14:08:43.176 INFO:tasks.workunit.client.0.smithi027.stdout: "persistent": [ 2020-09-12T14:08:43.176 INFO:tasks.workunit.client.0.smithi027.stdout: "kraken", 2020-09-12T14:08:43.176 INFO:tasks.workunit.client.0.smithi027.stdout: "luminous", 2020-09-12T14:08:43.176 INFO:tasks.workunit.client.0.smithi027.stdout: "mimic", 2020-09-12T14:08:43.177 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap-prune", 2020-09-12T14:08:43.177 INFO:tasks.workunit.client.0.smithi027.stdout: "nautilus", 2020-09-12T14:08:43.177 INFO:tasks.workunit.client.0.smithi027.stdout: "octopus", 2020-09-12T14:08:43.177 INFO:tasks.workunit.client.0.smithi027.stdout: "pacific" 2020-09-12T14:08:43.177 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:08:43.177 INFO:tasks.workunit.client.0.smithi027.stdout: "optional": [] 2020-09-12T14:08:43.177 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.177 INFO:tasks.workunit.client.0.smithi027.stdout: "mons": [ 2020-09-12T14:08:43.178 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.178 INFO:tasks.workunit.client.0.smithi027.stdout: "rank": 0, 2020-09-12T14:08:43.178 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "a", 2020-09-12T14:08:43.178 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:08:43.178 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:08:43.178 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.178 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:08:43.178 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:7115", 2020-09-12T14:08:43.179 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 0 2020-09-12T14:08:43.179 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.179 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.179 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.179 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:7115/0", 2020-09-12T14:08:43.179 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:7115/0", 2020-09-12T14:08:43.179 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 0, 2020-09-12T14:08:43.180 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 0 2020-09-12T14:08:43.180 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.180 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.180 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.180 INFO:tasks.workunit.client.0.smithi027.stdout: "quorum": [ 2020-09-12T14:08:43.180 INFO:tasks.workunit.client.0.smithi027.stdout: 0 2020-09-12T14:08:43.180 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:08:43.180 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap": { 2020-09-12T14:08:43.181 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 47, 2020-09-12T14:08:43.181 INFO:tasks.workunit.client.0.smithi027.stdout: "fsid": "7e90bad3-7973-4a8c-be65-6326ab931fe8", 2020-09-12T14:08:43.181 INFO:tasks.workunit.client.0.smithi027.stdout: "created": "2020-09-12T14:05:44.793120+0000", 2020-09-12T14:08:43.181 INFO:tasks.workunit.client.0.smithi027.stdout: "modified": "2020-09-12T14:08:05.415050+0000", 2020-09-12T14:08:43.181 INFO:tasks.workunit.client.0.smithi027.stdout: "last_up_change": "2020-09-12T14:08:04.877799+0000", 2020-09-12T14:08:43.181 INFO:tasks.workunit.client.0.smithi027.stdout: "last_in_change": "2020-09-12T14:07:16.077966+0000", 2020-09-12T14:08:43.181 INFO:tasks.workunit.client.0.smithi027.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-09-12T14:08:43.181 INFO:tasks.workunit.client.0.smithi027.stdout: "flags_num": 5800384, 2020-09-12T14:08:43.182 INFO:tasks.workunit.client.0.smithi027.stdout: "flags_set": [ 2020-09-12T14:08:43.182 INFO:tasks.workunit.client.0.smithi027.stdout: "nodown", 2020-09-12T14:08:43.182 INFO:tasks.workunit.client.0.smithi027.stdout: "noin", 2020-09-12T14:08:43.182 INFO:tasks.workunit.client.0.smithi027.stdout: "noout", 2020-09-12T14:08:43.182 INFO:tasks.workunit.client.0.smithi027.stdout: "pglog_hardlimit", 2020-09-12T14:08:43.182 INFO:tasks.workunit.client.0.smithi027.stdout: "purged_snapdirs", 2020-09-12T14:08:43.182 INFO:tasks.workunit.client.0.smithi027.stdout: "recovery_deletes", 2020-09-12T14:08:43.182 INFO:tasks.workunit.client.0.smithi027.stdout: "sortbitwise" 2020-09-12T14:08:43.183 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:08:43.183 INFO:tasks.workunit.client.0.smithi027.stdout: "crush_version": 8, 2020-09-12T14:08:43.183 INFO:tasks.workunit.client.0.smithi027.stdout: "full_ratio": 0.99000000953674316, 2020-09-12T14:08:43.183 INFO:tasks.workunit.client.0.smithi027.stdout: "backfillfull_ratio": 0.99000000953674316, 2020-09-12T14:08:43.183 INFO:tasks.workunit.client.0.smithi027.stdout: "nearfull_ratio": 0.85000002384185791, 2020-09-12T14:08:43.183 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_snapshot": "", 2020-09-12T14:08:43.183 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_max": 1, 2020-09-12T14:08:43.184 INFO:tasks.workunit.client.0.smithi027.stdout: "max_osd": 3, 2020-09-12T14:08:43.184 INFO:tasks.workunit.client.0.smithi027.stdout: "require_min_compat_client": "luminous", 2020-09-12T14:08:43.184 INFO:tasks.workunit.client.0.smithi027.stdout: "min_compat_client": "jewel", 2020-09-12T14:08:43.184 INFO:tasks.workunit.client.0.smithi027.stdout: "require_osd_release": "pacific", 2020-09-12T14:08:43.184 INFO:tasks.workunit.client.0.smithi027.stdout: "pools": [ 2020-09-12T14:08:43.184 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.184 INFO:tasks.workunit.client.0.smithi027.stdout: "pool": 1, 2020-09-12T14:08:43.184 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_name": "test", 2020-09-12T14:08:43.185 INFO:tasks.workunit.client.0.smithi027.stdout: "create_time": "2020-09-12T14:06:12.930962+0000", 2020-09-12T14:08:43.185 INFO:tasks.workunit.client.0.smithi027.stdout: "flags": 16389, 2020-09-12T14:08:43.185 INFO:tasks.workunit.client.0.smithi027.stdout: "flags_names": "hashpspool,ec_overwrites,pool_snaps", 2020-09-12T14:08:43.185 INFO:tasks.workunit.client.0.smithi027.stdout: "type": 3, 2020-09-12T14:08:43.185 INFO:tasks.workunit.client.0.smithi027.stdout: "size": 3, 2020-09-12T14:08:43.185 INFO:tasks.workunit.client.0.smithi027.stdout: "min_size": 2, 2020-09-12T14:08:43.185 INFO:tasks.workunit.client.0.smithi027.stdout: "crush_rule": 1, 2020-09-12T14:08:43.185 INFO:tasks.workunit.client.0.smithi027.stdout: "object_hash": 2, 2020-09-12T14:08:43.186 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_autoscale_mode": "off", 2020-09-12T14:08:43.186 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_num": 1, 2020-09-12T14:08:43.186 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_placement_num": 1, 2020-09-12T14:08:43.186 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_placement_num_target": 1, 2020-09-12T14:08:43.186 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_num_target": 1, 2020-09-12T14:08:43.186 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_num_pending": 1, 2020-09-12T14:08:43.186 INFO:tasks.workunit.client.0.smithi027.stdout: "last_pg_merge_meta": { 2020-09-12T14:08:43.187 INFO:tasks.workunit.client.0.smithi027.stdout: "source_pgid": "0.0", 2020-09-12T14:08:43.187 INFO:tasks.workunit.client.0.smithi027.stdout: "ready_epoch": 0, 2020-09-12T14:08:43.187 INFO:tasks.workunit.client.0.smithi027.stdout: "last_epoch_started": 0, 2020-09-12T14:08:43.187 INFO:tasks.workunit.client.0.smithi027.stdout: "last_epoch_clean": 0, 2020-09-12T14:08:43.187 INFO:tasks.workunit.client.0.smithi027.stdout: "source_version": "0'0", 2020-09-12T14:08:43.187 INFO:tasks.workunit.client.0.smithi027.stdout: "target_version": "0'0" 2020-09-12T14:08:43.187 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.187 INFO:tasks.workunit.client.0.smithi027.stdout: "last_change": "30", 2020-09-12T14:08:43.188 INFO:tasks.workunit.client.0.smithi027.stdout: "last_force_op_resend": "0", 2020-09-12T14:08:43.188 INFO:tasks.workunit.client.0.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2020-09-12T14:08:43.188 INFO:tasks.workunit.client.0.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2020-09-12T14:08:43.188 INFO:tasks.workunit.client.0.smithi027.stdout: "auid": 0, 2020-09-12T14:08:43.188 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_mode": "pool", 2020-09-12T14:08:43.188 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_seq": 1, 2020-09-12T14:08:43.188 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_epoch": 30, 2020-09-12T14:08:43.188 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_snaps": [ 2020-09-12T14:08:43.189 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.189 INFO:tasks.workunit.client.0.smithi027.stdout: "snapid": 1, 2020-09-12T14:08:43.189 INFO:tasks.workunit.client.0.smithi027.stdout: "stamp": "2020-09-12T14:06:56.423813+0000", 2020-09-12T14:08:43.189 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "snap1" 2020-09-12T14:08:43.189 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.189 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:08:43.189 INFO:tasks.workunit.client.0.smithi027.stdout: "removed_snaps": "[]", 2020-09-12T14:08:43.189 INFO:tasks.workunit.client.0.smithi027.stdout: "quota_max_bytes": 0, 2020-09-12T14:08:43.190 INFO:tasks.workunit.client.0.smithi027.stdout: "quota_max_objects": 0, 2020-09-12T14:08:43.190 INFO:tasks.workunit.client.0.smithi027.stdout: "tiers": [], 2020-09-12T14:08:43.190 INFO:tasks.workunit.client.0.smithi027.stdout: "tier_of": -1, 2020-09-12T14:08:43.190 INFO:tasks.workunit.client.0.smithi027.stdout: "read_tier": -1, 2020-09-12T14:08:43.190 INFO:tasks.workunit.client.0.smithi027.stdout: "write_tier": -1, 2020-09-12T14:08:43.190 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_mode": "none", 2020-09-12T14:08:43.190 INFO:tasks.workunit.client.0.smithi027.stdout: "target_max_bytes": 0, 2020-09-12T14:08:43.191 INFO:tasks.workunit.client.0.smithi027.stdout: "target_max_objects": 0, 2020-09-12T14:08:43.191 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-09-12T14:08:43.191 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-09-12T14:08:43.191 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2020-09-12T14:08:43.191 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_min_flush_age": 0, 2020-09-12T14:08:43.191 INFO:tasks.workunit.client.0.smithi027.stdout: "cache_min_evict_age": 0, 2020-09-12T14:08:43.191 INFO:tasks.workunit.client.0.smithi027.stdout: "erasure_code_profile": "myprofile", 2020-09-12T14:08:43.191 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_params": { 2020-09-12T14:08:43.192 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "none" 2020-09-12T14:08:43.192 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.192 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_period": 0, 2020-09-12T14:08:43.192 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_count": 0, 2020-09-12T14:08:43.192 INFO:tasks.workunit.client.0.smithi027.stdout: "use_gmt_hitset": true, 2020-09-12T14:08:43.192 INFO:tasks.workunit.client.0.smithi027.stdout: "min_read_recency_for_promote": 0, 2020-09-12T14:08:43.192 INFO:tasks.workunit.client.0.smithi027.stdout: "min_write_recency_for_promote": 0, 2020-09-12T14:08:43.192 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2020-09-12T14:08:43.193 INFO:tasks.workunit.client.0.smithi027.stdout: "hit_set_search_last_n": 0, 2020-09-12T14:08:43.193 INFO:tasks.workunit.client.0.smithi027.stdout: "grade_table": [], 2020-09-12T14:08:43.193 INFO:tasks.workunit.client.0.smithi027.stdout: "stripe_width": 8192, 2020-09-12T14:08:43.193 INFO:tasks.workunit.client.0.smithi027.stdout: "expected_num_objects": 0, 2020-09-12T14:08:43.193 INFO:tasks.workunit.client.0.smithi027.stdout: "fast_read": false, 2020-09-12T14:08:43.193 INFO:tasks.workunit.client.0.smithi027.stdout: "options": {}, 2020-09-12T14:08:43.193 INFO:tasks.workunit.client.0.smithi027.stdout: "application_metadata": {} 2020-09-12T14:08:43.193 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.194 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:08:43.194 INFO:tasks.workunit.client.0.smithi027.stdout: "osds": [ 2020-09-12T14:08:43.194 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.194 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:08:43.194 INFO:tasks.workunit.client.0.smithi027.stdout: "uuid": "3d802381-3749-43a0-ac4c-e7fa2730ba5f", 2020-09-12T14:08:43.194 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 1, 2020-09-12T14:08:43.194 INFO:tasks.workunit.client.0.smithi027.stdout: "in": 1, 2020-09-12T14:08:43.195 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 1, 2020-09-12T14:08:43.195 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_affinity": 1, 2020-09-12T14:08:43.195 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_begin": 36, 2020-09-12T14:08:43.195 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_end": 40, 2020-09-12T14:08:43.195 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 42, 2020-09-12T14:08:43.195 INFO:tasks.workunit.client.0.smithi027.stdout: "up_thru": 45, 2020-09-12T14:08:43.195 INFO:tasks.workunit.client.0.smithi027.stdout: "down_at": 41, 2020-09-12T14:08:43.195 INFO:tasks.workunit.client.0.smithi027.stdout: "lost_at": 0, 2020-09-12T14:08:43.196 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:08:43.196 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:08:43.196 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.196 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:08:43.196 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6802", 2020-09-12T14:08:43.196 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 63138 2020-09-12T14:08:43.196 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.196 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.197 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:08:43.197 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6803", 2020-09-12T14:08:43.197 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 63138 2020-09-12T14:08:43.197 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.197 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.197 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.197 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addrs": { 2020-09-12T14:08:43.198 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:08:43.198 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.198 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:08:43.198 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6804", 2020-09-12T14:08:43.198 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 63138 2020-09-12T14:08:43.198 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.198 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.198 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:08:43.199 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6805", 2020-09-12T14:08:43.199 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 63138 2020-09-12T14:08:43.199 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.199 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.199 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.199 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addrs": { 2020-09-12T14:08:43.199 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:08:43.199 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.200 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:08:43.200 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6808", 2020-09-12T14:08:43.200 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 63138 2020-09-12T14:08:43.200 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.200 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.200 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:08:43.200 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6809", 2020-09-12T14:08:43.200 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 63138 2020-09-12T14:08:43.201 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.201 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.201 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.201 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addrs": { 2020-09-12T14:08:43.201 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:08:43.201 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.201 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:08:43.201 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6806", 2020-09-12T14:08:43.202 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 63138 2020-09-12T14:08:43.202 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.202 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.202 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:08:43.202 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6807", 2020-09-12T14:08:43.202 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 63138 2020-09-12T14:08:43.202 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.203 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.203 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.203 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:6803/63138", 2020-09-12T14:08:43.203 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addr": "127.0.0.1:6805/63138", 2020-09-12T14:08:43.203 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addr": "127.0.0.1:6809/63138", 2020-09-12T14:08:43.203 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addr": "127.0.0.1:6807/63138", 2020-09-12T14:08:43.203 INFO:tasks.workunit.client.0.smithi027.stdout: "state": [ 2020-09-12T14:08:43.203 INFO:tasks.workunit.client.0.smithi027.stdout: "exists", 2020-09-12T14:08:43.204 INFO:tasks.workunit.client.0.smithi027.stdout: "up" 2020-09-12T14:08:43.204 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.204 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.204 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.204 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:08:43.204 INFO:tasks.workunit.client.0.smithi027.stdout: "uuid": "270594ae-5b9c-4b0c-bd1d-3fddd2a993d0", 2020-09-12T14:08:43.204 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 0, 2020-09-12T14:08:43.204 INFO:tasks.workunit.client.0.smithi027.stdout: "in": 1, 2020-09-12T14:08:43.205 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 1, 2020-09-12T14:08:43.205 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_affinity": 1, 2020-09-12T14:08:43.205 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_begin": 0, 2020-09-12T14:08:43.205 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_end": 0, 2020-09-12T14:08:43.205 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 10, 2020-09-12T14:08:43.205 INFO:tasks.workunit.client.0.smithi027.stdout: "up_thru": 25, 2020-09-12T14:08:43.205 INFO:tasks.workunit.client.0.smithi027.stdout: "down_at": 32, 2020-09-12T14:08:43.205 INFO:tasks.workunit.client.0.smithi027.stdout: "lost_at": 0, 2020-09-12T14:08:43.206 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:08:43.206 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:08:43.206 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.206 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:08:43.206 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6810", 2020-09-12T14:08:43.206 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 55733 2020-09-12T14:08:43.206 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.207 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.207 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:08:43.207 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6811", 2020-09-12T14:08:43.207 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 55733 2020-09-12T14:08:43.207 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.207 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.207 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.207 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addrs": { 2020-09-12T14:08:43.208 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:08:43.208 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.208 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:08:43.208 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6812", 2020-09-12T14:08:43.208 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 55733 2020-09-12T14:08:43.208 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.208 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.209 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:08:43.209 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6813", 2020-09-12T14:08:43.209 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 55733 2020-09-12T14:08:43.209 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.209 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.209 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.209 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addrs": { 2020-09-12T14:08:43.209 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:08:43.210 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.210 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:08:43.210 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6816", 2020-09-12T14:08:43.210 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 55733 2020-09-12T14:08:43.210 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.210 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.210 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:08:43.210 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6817", 2020-09-12T14:08:43.211 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 55733 2020-09-12T14:08:43.211 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.211 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.211 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.211 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addrs": { 2020-09-12T14:08:43.211 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:08:43.211 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.211 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:08:43.212 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6814", 2020-09-12T14:08:43.212 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 55733 2020-09-12T14:08:43.212 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.212 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.212 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:08:43.212 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6815", 2020-09-12T14:08:43.212 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 55733 2020-09-12T14:08:43.212 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.213 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.213 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.213 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:6811/55733", 2020-09-12T14:08:43.213 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addr": "127.0.0.1:6813/55733", 2020-09-12T14:08:43.213 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addr": "127.0.0.1:6817/55733", 2020-09-12T14:08:43.213 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addr": "127.0.0.1:6815/55733", 2020-09-12T14:08:43.213 INFO:tasks.workunit.client.0.smithi027.stdout: "state": [ 2020-09-12T14:08:43.214 INFO:tasks.workunit.client.0.smithi027.stdout: "exists" 2020-09-12T14:08:43.214 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.214 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.214 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.214 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:08:43.214 INFO:tasks.workunit.client.0.smithi027.stdout: "uuid": "ed2b134c-2770-4bad-8557-6c53d9f7b711", 2020-09-12T14:08:43.214 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 1, 2020-09-12T14:08:43.214 INFO:tasks.workunit.client.0.smithi027.stdout: "in": 1, 2020-09-12T14:08:43.215 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 1, 2020-09-12T14:08:43.215 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_affinity": 1, 2020-09-12T14:08:43.215 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_begin": 37, 2020-09-12T14:08:43.215 INFO:tasks.workunit.client.0.smithi027.stdout: "last_clean_end": 40, 2020-09-12T14:08:43.215 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 45, 2020-09-12T14:08:43.215 INFO:tasks.workunit.client.0.smithi027.stdout: "up_thru": 0, 2020-09-12T14:08:43.215 INFO:tasks.workunit.client.0.smithi027.stdout: "down_at": 44, 2020-09-12T14:08:43.215 INFO:tasks.workunit.client.0.smithi027.stdout: "lost_at": 0, 2020-09-12T14:08:43.216 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addrs": { 2020-09-12T14:08:43.216 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:08:43.216 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.216 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:08:43.216 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6810", 2020-09-12T14:08:43.216 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 63787 2020-09-12T14:08:43.216 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.216 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.217 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:08:43.217 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6811", 2020-09-12T14:08:43.217 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 63787 2020-09-12T14:08:43.217 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.217 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.217 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.217 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addrs": { 2020-09-12T14:08:43.218 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:08:43.218 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.218 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:08:43.218 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6812", 2020-09-12T14:08:43.218 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 63787 2020-09-12T14:08:43.218 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.218 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.218 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:08:43.219 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6813", 2020-09-12T14:08:43.219 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 63787 2020-09-12T14:08:43.219 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.219 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.219 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.219 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addrs": { 2020-09-12T14:08:43.219 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:08:43.219 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.220 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:08:43.220 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6816", 2020-09-12T14:08:43.220 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 63787 2020-09-12T14:08:43.220 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.220 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.220 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:08:43.220 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6817", 2020-09-12T14:08:43.220 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 63787 2020-09-12T14:08:43.221 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.221 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.221 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.221 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addrs": { 2020-09-12T14:08:43.221 INFO:tasks.workunit.client.0.smithi027.stdout: "addrvec": [ 2020-09-12T14:08:43.221 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.221 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v2", 2020-09-12T14:08:43.221 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6814", 2020-09-12T14:08:43.222 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 63787 2020-09-12T14:08:43.222 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.222 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.222 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "v1", 2020-09-12T14:08:43.222 INFO:tasks.workunit.client.0.smithi027.stdout: "addr": "127.0.0.1:6815", 2020-09-12T14:08:43.222 INFO:tasks.workunit.client.0.smithi027.stdout: "nonce": 63787 2020-09-12T14:08:43.222 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.223 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.223 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.223 INFO:tasks.workunit.client.0.smithi027.stdout: "public_addr": "127.0.0.1:6811/63787", 2020-09-12T14:08:43.223 INFO:tasks.workunit.client.0.smithi027.stdout: "cluster_addr": "127.0.0.1:6813/63787", 2020-09-12T14:08:43.223 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_back_addr": "127.0.0.1:6817/63787", 2020-09-12T14:08:43.223 INFO:tasks.workunit.client.0.smithi027.stdout: "heartbeat_front_addr": "127.0.0.1:6815/63787", 2020-09-12T14:08:43.223 INFO:tasks.workunit.client.0.smithi027.stdout: "state": [ 2020-09-12T14:08:43.223 INFO:tasks.workunit.client.0.smithi027.stdout: "exists", 2020-09-12T14:08:43.224 INFO:tasks.workunit.client.0.smithi027.stdout: "up" 2020-09-12T14:08:43.224 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.224 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.224 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:08:43.224 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_xinfo": [ 2020-09-12T14:08:43.224 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.224 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:08:43.224 INFO:tasks.workunit.client.0.smithi027.stdout: "down_stamp": "2020-09-12T14:08:02.118200+0000", 2020-09-12T14:08:43.225 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_probability": 0, 2020-09-12T14:08:43.225 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_interval": 0, 2020-09-12T14:08:43.225 INFO:tasks.workunit.client.0.smithi027.stdout: "features": 4540138292841938943, 2020-09-12T14:08:43.225 INFO:tasks.workunit.client.0.smithi027.stdout: "old_weight": 0, 2020-09-12T14:08:43.225 INFO:tasks.workunit.client.0.smithi027.stdout: "last_purged_snaps_scrub": "2020-09-12T14:05:53.422495+0000", 2020-09-12T14:08:43.225 INFO:tasks.workunit.client.0.smithi027.stdout: "dead_epoch": 0 2020-09-12T14:08:43.225 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.225 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.226 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:08:43.226 INFO:tasks.workunit.client.0.smithi027.stdout: "down_stamp": "2020-09-12T14:07:08.150640+0000", 2020-09-12T14:08:43.226 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_probability": 0, 2020-09-12T14:08:43.226 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_interval": 0, 2020-09-12T14:08:43.226 INFO:tasks.workunit.client.0.smithi027.stdout: "features": 4540138292841938943, 2020-09-12T14:08:43.226 INFO:tasks.workunit.client.0.smithi027.stdout: "old_weight": 0, 2020-09-12T14:08:43.226 INFO:tasks.workunit.client.0.smithi027.stdout: "last_purged_snaps_scrub": "2020-09-12T14:06:01.399666+0000", 2020-09-12T14:08:43.227 INFO:tasks.workunit.client.0.smithi027.stdout: "dead_epoch": 0 2020-09-12T14:08:43.227 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.227 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.227 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:08:43.227 INFO:tasks.workunit.client.0.smithi027.stdout: "down_stamp": "2020-09-12T14:08:04.642602+0000", 2020-09-12T14:08:43.227 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_probability": 0, 2020-09-12T14:08:43.227 INFO:tasks.workunit.client.0.smithi027.stdout: "laggy_interval": 0, 2020-09-12T14:08:43.227 INFO:tasks.workunit.client.0.smithi027.stdout: "features": 4540138292841938943, 2020-09-12T14:08:43.228 INFO:tasks.workunit.client.0.smithi027.stdout: "old_weight": 0, 2020-09-12T14:08:43.228 INFO:tasks.workunit.client.0.smithi027.stdout: "last_purged_snaps_scrub": "2020-09-12T14:06:09.531147+0000", 2020-09-12T14:08:43.228 INFO:tasks.workunit.client.0.smithi027.stdout: "dead_epoch": 0 2020-09-12T14:08:43.228 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.228 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:08:43.228 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_upmap": [], 2020-09-12T14:08:43.228 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_upmap_items": [], 2020-09-12T14:08:43.229 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_temp": [], 2020-09-12T14:08:43.229 INFO:tasks.workunit.client.0.smithi027.stdout: "primary_temp": [], 2020-09-12T14:08:43.229 INFO:tasks.workunit.client.0.smithi027.stdout: "blocklist": {}, 2020-09-12T14:08:43.229 INFO:tasks.workunit.client.0.smithi027.stdout: "erasure_code_profiles": { 2020-09-12T14:08:43.229 INFO:tasks.workunit.client.0.smithi027.stdout: "default": { 2020-09-12T14:08:43.229 INFO:tasks.workunit.client.0.smithi027.stdout: "k": "2", 2020-09-12T14:08:43.229 INFO:tasks.workunit.client.0.smithi027.stdout: "m": "2", 2020-09-12T14:08:43.229 INFO:tasks.workunit.client.0.smithi027.stdout: "plugin": "jerasure", 2020-09-12T14:08:43.230 INFO:tasks.workunit.client.0.smithi027.stdout: "technique": "reed_sol_van" 2020-09-12T14:08:43.230 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.230 INFO:tasks.workunit.client.0.smithi027.stdout: "myprofile": { 2020-09-12T14:08:43.230 INFO:tasks.workunit.client.0.smithi027.stdout: "crush-device-class": "", 2020-09-12T14:08:43.230 INFO:tasks.workunit.client.0.smithi027.stdout: "crush-failure-domain": "osd", 2020-09-12T14:08:43.230 INFO:tasks.workunit.client.0.smithi027.stdout: "crush-root": "default", 2020-09-12T14:08:43.230 INFO:tasks.workunit.client.0.smithi027.stdout: "jerasure-per-chunk-alignment": "false", 2020-09-12T14:08:43.230 INFO:tasks.workunit.client.0.smithi027.stdout: "k": "2", 2020-09-12T14:08:43.231 INFO:tasks.workunit.client.0.smithi027.stdout: "m": "1", 2020-09-12T14:08:43.231 INFO:tasks.workunit.client.0.smithi027.stdout: "plugin": "jerasure", 2020-09-12T14:08:43.231 INFO:tasks.workunit.client.0.smithi027.stdout: "technique": "reed_sol_van", 2020-09-12T14:08:43.231 INFO:tasks.workunit.client.0.smithi027.stdout: "w": "8" 2020-09-12T14:08:43.231 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.231 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.231 INFO:tasks.workunit.client.0.smithi027.stdout: "removed_snaps_queue": [], 2020-09-12T14:08:43.231 INFO:tasks.workunit.client.0.smithi027.stdout: "new_removed_snaps": [], 2020-09-12T14:08:43.232 INFO:tasks.workunit.client.0.smithi027.stdout: "new_purged_snaps": [], 2020-09-12T14:08:43.232 INFO:tasks.workunit.client.0.smithi027.stdout: "crush_node_flags": {}, 2020-09-12T14:08:43.232 INFO:tasks.workunit.client.0.smithi027.stdout: "device_class_flags": {} 2020-09-12T14:08:43.232 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.232 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_metadata": [ 2020-09-12T14:08:43.232 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.232 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:08:43.232 INFO:tasks.workunit.client.0.smithi027.stdout: "arch": "x86_64", 2020-09-12T14:08:43.233 INFO:tasks.workunit.client.0.smithi027.stdout: "back_addr": "[v2:127.0.0.1:6804/63138,v1:127.0.0.1:6805/63138]", 2020-09-12T14:08:43.233 INFO:tasks.workunit.client.0.smithi027.stdout: "back_iface": "", 2020-09-12T14:08:43.233 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs": "1", 2020-09-12T14:08:43.233 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_db": "0", 2020-09-12T14:08:43.233 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_wal": "0", 2020-09-12T14:08:43.233 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_single_shared_device": "1", 2020-09-12T14:08:43.233 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_access_mode": "file", 2020-09-12T14:08:43.234 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_block_size": "4096", 2020-09-12T14:08:43.234 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_devices": "sda", 2020-09-12T14:08:43.234 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_driver": "KernelDevice", 2020-09-12T14:08:43.234 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2020-09-12T14:08:43.234 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_rotational": "1", 2020-09-12T14:08:43.234 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_size": "107374182400", 2020-09-12T14:08:43.234 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_support_discard": "0", 2020-09-12T14:08:43.234 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_type": "hdd", 2020-09-12T14:08:43.235 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_release": "pacific", 2020-09-12T14:08:43.235 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version": "ceph version 16.0.0-5485-gf346cc73d15 (f346cc73d15a568d3bec1c99a58953de76f57b85) pacific (dev)", 2020-09-12T14:08:43.235 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version_short": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:08:43.235 INFO:tasks.workunit.client.0.smithi027.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2020-09-12T14:08:43.235 INFO:tasks.workunit.client.0.smithi027.stdout: "default_device_class": "hdd", 2020-09-12T14:08:43.235 INFO:tasks.workunit.client.0.smithi027.stdout: "device_ids": "sda=ST1000NM0033-9ZM173_Z1W4H3N7", 2020-09-12T14:08:43.235 INFO:tasks.workunit.client.0.smithi027.stdout: "device_paths": "sda=/dev/disk/by-path/pci-0000:00:1f.2-ata-1", 2020-09-12T14:08:43.235 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": "sda", 2020-09-12T14:08:43.236 INFO:tasks.workunit.client.0.smithi027.stdout: "distro": "centos", 2020-09-12T14:08:43.236 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_description": "CentOS Linux 8 (Core)", 2020-09-12T14:08:43.236 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_version": "8", 2020-09-12T14:08:43.236 INFO:tasks.workunit.client.0.smithi027.stdout: "front_addr": "[v2:127.0.0.1:6802/63138,v1:127.0.0.1:6803/63138]", 2020-09-12T14:08:43.236 INFO:tasks.workunit.client.0.smithi027.stdout: "front_iface": "", 2020-09-12T14:08:43.236 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/63138,v1:127.0.0.1:6809/63138]", 2020-09-12T14:08:43.236 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/63138,v1:127.0.0.1:6807/63138]", 2020-09-12T14:08:43.236 INFO:tasks.workunit.client.0.smithi027.stdout: "hostname": "smithi027", 2020-09-12T14:08:43.237 INFO:tasks.workunit.client.0.smithi027.stdout: "journal_rotational": "1", 2020-09-12T14:08:43.237 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_description": "#1 SMP Sun Jul 26 03:54:29 UTC 2020", 2020-09-12T14:08:43.237 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_version": "4.18.0-193.14.2.el8_2.x86_64", 2020-09-12T14:08:43.237 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_swap_kb": "0", 2020-09-12T14:08:43.237 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_total_kb": "32659336", 2020-09-12T14:08:43.237 INFO:tasks.workunit.client.0.smithi027.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2020-09-12T14:08:43.237 INFO:tasks.workunit.client.0.smithi027.stdout: "objectstore_numa_unknown_devices": "sda", 2020-09-12T14:08:43.238 INFO:tasks.workunit.client.0.smithi027.stdout: "os": "Linux", 2020-09-12T14:08:43.238 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_data": "td/divergent-priors/0", 2020-09-12T14:08:43.238 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_objectstore": "bluestore", 2020-09-12T14:08:43.238 INFO:tasks.workunit.client.0.smithi027.stdout: "osdspec_affinity": "", 2020-09-12T14:08:43.238 INFO:tasks.workunit.client.0.smithi027.stdout: "rotational": "1" 2020-09-12T14:08:43.238 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.238 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.238 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:08:43.239 INFO:tasks.workunit.client.0.smithi027.stdout: "arch": "x86_64", 2020-09-12T14:08:43.239 INFO:tasks.workunit.client.0.smithi027.stdout: "back_addr": "[v2:127.0.0.1:6812/55733,v1:127.0.0.1:6813/55733]", 2020-09-12T14:08:43.239 INFO:tasks.workunit.client.0.smithi027.stdout: "back_iface": "", 2020-09-12T14:08:43.239 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs": "1", 2020-09-12T14:08:43.239 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_db": "0", 2020-09-12T14:08:43.239 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_wal": "0", 2020-09-12T14:08:43.239 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_single_shared_device": "1", 2020-09-12T14:08:43.239 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_access_mode": "file", 2020-09-12T14:08:43.240 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_block_size": "4096", 2020-09-12T14:08:43.240 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_devices": "sda", 2020-09-12T14:08:43.240 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_driver": "KernelDevice", 2020-09-12T14:08:43.240 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2020-09-12T14:08:43.240 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_rotational": "1", 2020-09-12T14:08:43.240 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_size": "107374182400", 2020-09-12T14:08:43.240 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_support_discard": "0", 2020-09-12T14:08:43.240 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_type": "hdd", 2020-09-12T14:08:43.241 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_release": "pacific", 2020-09-12T14:08:43.241 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version": "ceph version 16.0.0-5485-gf346cc73d15 (f346cc73d15a568d3bec1c99a58953de76f57b85) pacific (dev)", 2020-09-12T14:08:43.241 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version_short": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:08:43.241 INFO:tasks.workunit.client.0.smithi027.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2020-09-12T14:08:43.241 INFO:tasks.workunit.client.0.smithi027.stdout: "default_device_class": "hdd", 2020-09-12T14:08:43.241 INFO:tasks.workunit.client.0.smithi027.stdout: "device_ids": "sda=ST1000NM0033-9ZM173_Z1W4H3N7", 2020-09-12T14:08:43.241 INFO:tasks.workunit.client.0.smithi027.stdout: "device_paths": "sda=/dev/disk/by-path/pci-0000:00:1f.2-ata-1", 2020-09-12T14:08:43.242 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": "sda", 2020-09-12T14:08:43.242 INFO:tasks.workunit.client.0.smithi027.stdout: "distro": "centos", 2020-09-12T14:08:43.242 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_description": "CentOS Linux 8 (Core)", 2020-09-12T14:08:43.242 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_version": "8", 2020-09-12T14:08:43.242 INFO:tasks.workunit.client.0.smithi027.stdout: "front_addr": "[v2:127.0.0.1:6810/55733,v1:127.0.0.1:6811/55733]", 2020-09-12T14:08:43.242 INFO:tasks.workunit.client.0.smithi027.stdout: "front_iface": "", 2020-09-12T14:08:43.242 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/55733,v1:127.0.0.1:6817/55733]", 2020-09-12T14:08:43.242 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/55733,v1:127.0.0.1:6815/55733]", 2020-09-12T14:08:43.243 INFO:tasks.workunit.client.0.smithi027.stdout: "hostname": "smithi027", 2020-09-12T14:08:43.243 INFO:tasks.workunit.client.0.smithi027.stdout: "journal_rotational": "1", 2020-09-12T14:08:43.243 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_description": "#1 SMP Sun Jul 26 03:54:29 UTC 2020", 2020-09-12T14:08:43.243 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_version": "4.18.0-193.14.2.el8_2.x86_64", 2020-09-12T14:08:43.243 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_swap_kb": "0", 2020-09-12T14:08:43.243 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_total_kb": "32659336", 2020-09-12T14:08:43.243 INFO:tasks.workunit.client.0.smithi027.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2020-09-12T14:08:43.243 INFO:tasks.workunit.client.0.smithi027.stdout: "objectstore_numa_unknown_devices": "sda", 2020-09-12T14:08:43.244 INFO:tasks.workunit.client.0.smithi027.stdout: "os": "Linux", 2020-09-12T14:08:43.244 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_data": "td/divergent-priors/1", 2020-09-12T14:08:43.244 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_objectstore": "bluestore", 2020-09-12T14:08:43.244 INFO:tasks.workunit.client.0.smithi027.stdout: "osdspec_affinity": "", 2020-09-12T14:08:43.244 INFO:tasks.workunit.client.0.smithi027.stdout: "rotational": "1" 2020-09-12T14:08:43.244 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.244 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.244 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:08:43.245 INFO:tasks.workunit.client.0.smithi027.stdout: "arch": "x86_64", 2020-09-12T14:08:43.245 INFO:tasks.workunit.client.0.smithi027.stdout: "back_addr": "[v2:127.0.0.1:6812/63787,v1:127.0.0.1:6813/63787]", 2020-09-12T14:08:43.245 INFO:tasks.workunit.client.0.smithi027.stdout: "back_iface": "", 2020-09-12T14:08:43.245 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs": "1", 2020-09-12T14:08:43.245 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_db": "0", 2020-09-12T14:08:43.245 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_dedicated_wal": "0", 2020-09-12T14:08:43.245 INFO:tasks.workunit.client.0.smithi027.stdout: "bluefs_single_shared_device": "1", 2020-09-12T14:08:43.246 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_access_mode": "file", 2020-09-12T14:08:43.246 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_block_size": "4096", 2020-09-12T14:08:43.246 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_devices": "sda", 2020-09-12T14:08:43.246 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_driver": "KernelDevice", 2020-09-12T14:08:43.246 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2020-09-12T14:08:43.246 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_rotational": "1", 2020-09-12T14:08:43.246 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_size": "107374182400", 2020-09-12T14:08:43.246 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_support_discard": "0", 2020-09-12T14:08:43.247 INFO:tasks.workunit.client.0.smithi027.stdout: "bluestore_bdev_type": "hdd", 2020-09-12T14:08:43.247 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_release": "pacific", 2020-09-12T14:08:43.247 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version": "ceph version 16.0.0-5485-gf346cc73d15 (f346cc73d15a568d3bec1c99a58953de76f57b85) pacific (dev)", 2020-09-12T14:08:43.247 INFO:tasks.workunit.client.0.smithi027.stdout: "ceph_version_short": "16.0.0-5485-gf346cc73d15", 2020-09-12T14:08:43.247 INFO:tasks.workunit.client.0.smithi027.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2020-09-12T14:08:43.247 INFO:tasks.workunit.client.0.smithi027.stdout: "default_device_class": "hdd", 2020-09-12T14:08:43.247 INFO:tasks.workunit.client.0.smithi027.stdout: "device_ids": "sda=ST1000NM0033-9ZM173_Z1W4H3N7", 2020-09-12T14:08:43.247 INFO:tasks.workunit.client.0.smithi027.stdout: "device_paths": "sda=/dev/disk/by-path/pci-0000:00:1f.2-ata-1", 2020-09-12T14:08:43.248 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": "sda", 2020-09-12T14:08:43.248 INFO:tasks.workunit.client.0.smithi027.stdout: "distro": "centos", 2020-09-12T14:08:43.248 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_description": "CentOS Linux 8 (Core)", 2020-09-12T14:08:43.248 INFO:tasks.workunit.client.0.smithi027.stdout: "distro_version": "8", 2020-09-12T14:08:43.248 INFO:tasks.workunit.client.0.smithi027.stdout: "front_addr": "[v2:127.0.0.1:6810/63787,v1:127.0.0.1:6811/63787]", 2020-09-12T14:08:43.248 INFO:tasks.workunit.client.0.smithi027.stdout: "front_iface": "", 2020-09-12T14:08:43.248 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/63787,v1:127.0.0.1:6817/63787]", 2020-09-12T14:08:43.248 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/63787,v1:127.0.0.1:6815/63787]", 2020-09-12T14:08:43.249 INFO:tasks.workunit.client.0.smithi027.stdout: "hostname": "smithi027", 2020-09-12T14:08:43.249 INFO:tasks.workunit.client.0.smithi027.stdout: "journal_rotational": "1", 2020-09-12T14:08:43.249 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_description": "#1 SMP Sun Jul 26 03:54:29 UTC 2020", 2020-09-12T14:08:43.249 INFO:tasks.workunit.client.0.smithi027.stdout: "kernel_version": "4.18.0-193.14.2.el8_2.x86_64", 2020-09-12T14:08:43.249 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_swap_kb": "0", 2020-09-12T14:08:43.249 INFO:tasks.workunit.client.0.smithi027.stdout: "mem_total_kb": "32659336", 2020-09-12T14:08:43.249 INFO:tasks.workunit.client.0.smithi027.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2020-09-12T14:08:43.250 INFO:tasks.workunit.client.0.smithi027.stdout: "objectstore_numa_unknown_devices": "sda", 2020-09-12T14:08:43.250 INFO:tasks.workunit.client.0.smithi027.stdout: "os": "Linux", 2020-09-12T14:08:43.250 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_data": "td/divergent-priors/2", 2020-09-12T14:08:43.250 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_objectstore": "bluestore", 2020-09-12T14:08:43.250 INFO:tasks.workunit.client.0.smithi027.stdout: "osdspec_affinity": "", 2020-09-12T14:08:43.250 INFO:tasks.workunit.client.0.smithi027.stdout: "rotational": "1" 2020-09-12T14:08:43.250 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.250 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:08:43.251 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap_clean_epochs": { 2020-09-12T14:08:43.251 INFO:tasks.workunit.client.0.smithi027.stdout: "min_last_epoch_clean": 0, 2020-09-12T14:08:43.251 INFO:tasks.workunit.client.0.smithi027.stdout: "last_epoch_clean": { 2020-09-12T14:08:43.251 INFO:tasks.workunit.client.0.smithi027.stdout: "per_pool": [] 2020-09-12T14:08:43.251 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.251 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_epochs": [ 2020-09-12T14:08:43.251 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.251 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:08:43.252 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 43 2020-09-12T14:08:43.252 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.252 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.252 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:08:43.252 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 10 2020-09-12T14:08:43.252 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.252 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.253 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:08:43.253 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 46 2020-09-12T14:08:43.253 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.253 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.253 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.253 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap_first_committed": 1, 2020-09-12T14:08:43.253 INFO:tasks.workunit.client.0.smithi027.stdout: "osdmap_last_committed": 47, 2020-09-12T14:08:43.253 INFO:tasks.workunit.client.0.smithi027.stdout: "crushmap": { 2020-09-12T14:08:43.254 INFO:tasks.workunit.client.0.smithi027.stdout: "devices": [ 2020-09-12T14:08:43.254 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.254 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:08:43.254 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd.0", 2020-09-12T14:08:43.254 INFO:tasks.workunit.client.0.smithi027.stdout: "class": "hdd" 2020-09-12T14:08:43.254 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.254 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.254 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:08:43.255 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd.1", 2020-09-12T14:08:43.255 INFO:tasks.workunit.client.0.smithi027.stdout: "class": "hdd" 2020-09-12T14:08:43.255 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.255 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.255 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:08:43.255 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd.2", 2020-09-12T14:08:43.255 INFO:tasks.workunit.client.0.smithi027.stdout: "class": "hdd" 2020-09-12T14:08:43.256 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.256 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:08:43.256 INFO:tasks.workunit.client.0.smithi027.stdout: "types": [ 2020-09-12T14:08:43.256 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.256 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 0, 2020-09-12T14:08:43.256 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "osd" 2020-09-12T14:08:43.256 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.256 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.257 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 1, 2020-09-12T14:08:43.257 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "host" 2020-09-12T14:08:43.257 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.257 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.257 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 2, 2020-09-12T14:08:43.257 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "chassis" 2020-09-12T14:08:43.257 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.257 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.258 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 3, 2020-09-12T14:08:43.258 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "rack" 2020-09-12T14:08:43.258 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.258 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.258 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 4, 2020-09-12T14:08:43.258 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "row" 2020-09-12T14:08:43.258 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.259 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.259 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 5, 2020-09-12T14:08:43.259 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "pdu" 2020-09-12T14:08:43.259 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.259 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.259 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 6, 2020-09-12T14:08:43.259 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "pod" 2020-09-12T14:08:43.259 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.260 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.260 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 7, 2020-09-12T14:08:43.260 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "room" 2020-09-12T14:08:43.260 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.260 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.260 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 8, 2020-09-12T14:08:43.260 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "datacenter" 2020-09-12T14:08:43.260 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.261 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.261 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 9, 2020-09-12T14:08:43.261 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "zone" 2020-09-12T14:08:43.261 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.261 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.261 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 10, 2020-09-12T14:08:43.261 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "region" 2020-09-12T14:08:43.261 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.262 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.262 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 11, 2020-09-12T14:08:43.262 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "root" 2020-09-12T14:08:43.262 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.262 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:08:43.262 INFO:tasks.workunit.client.0.smithi027.stdout: "buckets": [ 2020-09-12T14:08:43.262 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.262 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -1, 2020-09-12T14:08:43.263 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "default", 2020-09-12T14:08:43.263 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 11, 2020-09-12T14:08:43.263 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "root", 2020-09-12T14:08:43.263 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:08:43.263 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:08:43.263 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:08:43.263 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:08:43.263 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.264 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -3, 2020-09-12T14:08:43.264 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:08:43.264 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:08:43.264 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.264 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.264 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.264 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.265 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -2, 2020-09-12T14:08:43.265 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "default~hdd", 2020-09-12T14:08:43.265 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 11, 2020-09-12T14:08:43.265 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "root", 2020-09-12T14:08:43.265 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:08:43.265 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:08:43.265 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:08:43.265 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:08:43.266 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.266 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -4, 2020-09-12T14:08:43.266 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:08:43.266 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:08:43.266 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.266 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.266 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.266 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.267 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -3, 2020-09-12T14:08:43.267 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "smithi027", 2020-09-12T14:08:43.267 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 1, 2020-09-12T14:08:43.267 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "host", 2020-09-12T14:08:43.267 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:08:43.267 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:08:43.267 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:08:43.268 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:08:43.268 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.268 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:08:43.268 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:08:43.268 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:08:43.268 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.268 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.268 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:08:43.269 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:08:43.269 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 1 2020-09-12T14:08:43.269 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.269 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.269 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:08:43.269 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:08:43.269 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 2 2020-09-12T14:08:43.269 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.270 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.270 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.270 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.270 INFO:tasks.workunit.client.0.smithi027.stdout: "id": -4, 2020-09-12T14:08:43.270 INFO:tasks.workunit.client.0.smithi027.stdout: "name": "smithi027~hdd", 2020-09-12T14:08:43.270 INFO:tasks.workunit.client.0.smithi027.stdout: "type_id": 1, 2020-09-12T14:08:43.270 INFO:tasks.workunit.client.0.smithi027.stdout: "type_name": "host", 2020-09-12T14:08:43.270 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 19206, 2020-09-12T14:08:43.271 INFO:tasks.workunit.client.0.smithi027.stdout: "alg": "straw2", 2020-09-12T14:08:43.271 INFO:tasks.workunit.client.0.smithi027.stdout: "hash": "rjenkins1", 2020-09-12T14:08:43.271 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:08:43.271 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.271 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 0, 2020-09-12T14:08:43.271 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:08:43.271 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 0 2020-09-12T14:08:43.272 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.272 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.272 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 1, 2020-09-12T14:08:43.272 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:08:43.272 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 1 2020-09-12T14:08:43.272 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.272 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.272 INFO:tasks.workunit.client.0.smithi027.stdout: "id": 2, 2020-09-12T14:08:43.273 INFO:tasks.workunit.client.0.smithi027.stdout: "weight": 6402, 2020-09-12T14:08:43.273 INFO:tasks.workunit.client.0.smithi027.stdout: "pos": 2 2020-09-12T14:08:43.273 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.273 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.273 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.273 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:08:43.273 INFO:tasks.workunit.client.0.smithi027.stdout: "rules": [ 2020-09-12T14:08:43.273 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.274 INFO:tasks.workunit.client.0.smithi027.stdout: "rule_id": 0, 2020-09-12T14:08:43.274 INFO:tasks.workunit.client.0.smithi027.stdout: "rule_name": "replicated_rule", 2020-09-12T14:08:43.274 INFO:tasks.workunit.client.0.smithi027.stdout: "ruleset": 0, 2020-09-12T14:08:43.274 INFO:tasks.workunit.client.0.smithi027.stdout: "type": 1, 2020-09-12T14:08:43.274 INFO:tasks.workunit.client.0.smithi027.stdout: "min_size": 1, 2020-09-12T14:08:43.274 INFO:tasks.workunit.client.0.smithi027.stdout: "max_size": 10, 2020-09-12T14:08:43.274 INFO:tasks.workunit.client.0.smithi027.stdout: "steps": [ 2020-09-12T14:08:43.274 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.275 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "take", 2020-09-12T14:08:43.275 INFO:tasks.workunit.client.0.smithi027.stdout: "item": -1, 2020-09-12T14:08:43.275 INFO:tasks.workunit.client.0.smithi027.stdout: "item_name": "default" 2020-09-12T14:08:43.275 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.275 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.275 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "choose_firstn", 2020-09-12T14:08:43.275 INFO:tasks.workunit.client.0.smithi027.stdout: "num": 0, 2020-09-12T14:08:43.276 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "osd" 2020-09-12T14:08:43.276 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.276 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.276 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "emit" 2020-09-12T14:08:43.276 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.276 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.276 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.276 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.277 INFO:tasks.workunit.client.0.smithi027.stdout: "rule_id": 1, 2020-09-12T14:08:43.277 INFO:tasks.workunit.client.0.smithi027.stdout: "rule_name": "test", 2020-09-12T14:08:43.277 INFO:tasks.workunit.client.0.smithi027.stdout: "ruleset": 1, 2020-09-12T14:08:43.277 INFO:tasks.workunit.client.0.smithi027.stdout: "type": 3, 2020-09-12T14:08:43.277 INFO:tasks.workunit.client.0.smithi027.stdout: "min_size": 3, 2020-09-12T14:08:43.277 INFO:tasks.workunit.client.0.smithi027.stdout: "max_size": 3, 2020-09-12T14:08:43.277 INFO:tasks.workunit.client.0.smithi027.stdout: "steps": [ 2020-09-12T14:08:43.277 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.278 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "set_chooseleaf_tries", 2020-09-12T14:08:43.278 INFO:tasks.workunit.client.0.smithi027.stdout: "num": 5 2020-09-12T14:08:43.278 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.278 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.278 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "set_choose_tries", 2020-09-12T14:08:43.278 INFO:tasks.workunit.client.0.smithi027.stdout: "num": 100 2020-09-12T14:08:43.278 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.278 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.279 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "take", 2020-09-12T14:08:43.279 INFO:tasks.workunit.client.0.smithi027.stdout: "item": -1, 2020-09-12T14:08:43.279 INFO:tasks.workunit.client.0.smithi027.stdout: "item_name": "default" 2020-09-12T14:08:43.279 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.279 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.279 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "choose_indep", 2020-09-12T14:08:43.279 INFO:tasks.workunit.client.0.smithi027.stdout: "num": 0, 2020-09-12T14:08:43.279 INFO:tasks.workunit.client.0.smithi027.stdout: "type": "osd" 2020-09-12T14:08:43.280 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.280 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.280 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "emit" 2020-09-12T14:08:43.280 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.280 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:08:43.280 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.280 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:08:43.281 INFO:tasks.workunit.client.0.smithi027.stdout: "tunables": { 2020-09-12T14:08:43.281 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_local_tries": 0, 2020-09-12T14:08:43.281 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_local_fallback_tries": 0, 2020-09-12T14:08:43.281 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_total_tries": 50, 2020-09-12T14:08:43.281 INFO:tasks.workunit.client.0.smithi027.stdout: "chooseleaf_descend_once": 1, 2020-09-12T14:08:43.281 INFO:tasks.workunit.client.0.smithi027.stdout: "chooseleaf_vary_r": 1, 2020-09-12T14:08:43.281 INFO:tasks.workunit.client.0.smithi027.stdout: "chooseleaf_stable": 1, 2020-09-12T14:08:43.281 INFO:tasks.workunit.client.0.smithi027.stdout: "straw_calc_version": 1, 2020-09-12T14:08:43.282 INFO:tasks.workunit.client.0.smithi027.stdout: "allowed_bucket_algs": 54, 2020-09-12T14:08:43.282 INFO:tasks.workunit.client.0.smithi027.stdout: "profile": "jewel", 2020-09-12T14:08:43.282 INFO:tasks.workunit.client.0.smithi027.stdout: "optimal_tunables": 1, 2020-09-12T14:08:43.282 INFO:tasks.workunit.client.0.smithi027.stdout: "legacy_tunables": 0, 2020-09-12T14:08:43.282 INFO:tasks.workunit.client.0.smithi027.stdout: "minimum_required_version": "jewel", 2020-09-12T14:08:43.282 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables": 1, 2020-09-12T14:08:43.282 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables2": 1, 2020-09-12T14:08:43.282 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v2_rules": 1, 2020-09-12T14:08:43.283 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables3": 1, 2020-09-12T14:08:43.283 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v3_rules": 0, 2020-09-12T14:08:43.283 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v4_buckets": 1, 2020-09-12T14:08:43.283 INFO:tasks.workunit.client.0.smithi027.stdout: "require_feature_tunables5": 1, 2020-09-12T14:08:43.283 INFO:tasks.workunit.client.0.smithi027.stdout: "has_v5_rules": 0 2020-09-12T14:08:43.283 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.283 INFO:tasks.workunit.client.0.smithi027.stdout: "choose_args": {} 2020-09-12T14:08:43.283 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.284 INFO:tasks.workunit.client.0.smithi027.stdout: "fsmap": { 2020-09-12T14:08:43.284 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 1, 2020-09-12T14:08:43.284 INFO:tasks.workunit.client.0.smithi027.stdout: "default_fscid": -1, 2020-09-12T14:08:43.284 INFO:tasks.workunit.client.0.smithi027.stdout: "compat": { 2020-09-12T14:08:43.284 INFO:tasks.workunit.client.0.smithi027.stdout: "compat": {}, 2020-09-12T14:08:43.284 INFO:tasks.workunit.client.0.smithi027.stdout: "ro_compat": {}, 2020-09-12T14:08:43.284 INFO:tasks.workunit.client.0.smithi027.stdout: "incompat": { 2020-09-12T14:08:43.285 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_1": "base v0.20", 2020-09-12T14:08:43.285 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_2": "client writeable ranges", 2020-09-12T14:08:43.285 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_3": "default file layouts on dirs", 2020-09-12T14:08:43.285 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_4": "dir inode in separate object", 2020-09-12T14:08:43.285 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_5": "mds uses versioned encoding", 2020-09-12T14:08:43.285 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_6": "dirfrag is stored in omap", 2020-09-12T14:08:43.285 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_8": "no anchor table", 2020-09-12T14:08:43.285 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_9": "file layout v2", 2020-09-12T14:08:43.286 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_10": "snaprealm v2" 2020-09-12T14:08:43.286 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.286 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.286 INFO:tasks.workunit.client.0.smithi027.stdout: "feature_flags": { 2020-09-12T14:08:43.286 INFO:tasks.workunit.client.0.smithi027.stdout: "enable_multiple": false, 2020-09-12T14:08:43.286 INFO:tasks.workunit.client.0.smithi027.stdout: "ever_enabled_multiple": false 2020-09-12T14:08:43.286 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.286 INFO:tasks.workunit.client.0.smithi027.stdout: "standbys": [], 2020-09-12T14:08:43.287 INFO:tasks.workunit.client.0.smithi027.stdout: "filesystems": [] 2020-09-12T14:08:43.287 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.287 INFO:tasks.workunit.client.0.smithi027.stdout: "mdsmap_first_committed": 1, 2020-09-12T14:08:43.287 INFO:tasks.workunit.client.0.smithi027.stdout: "mdsmap_last_committed": 1, 2020-09-12T14:08:43.287 INFO:tasks.workunit.client.0.smithi027.stdout: "auth": { 2020-09-12T14:08:43.287 INFO:tasks.workunit.client.0.smithi027.stdout: "first_committed": 1, 2020-09-12T14:08:43.287 INFO:tasks.workunit.client.0.smithi027.stdout: "last_committed": 5, 2020-09-12T14:08:43.287 INFO:tasks.workunit.client.0.smithi027.stdout: "num_secrets": 3 2020-09-12T14:08:43.288 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.288 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg": 1, 2020-09-12T14:08:43.288 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_active": 1, 2020-09-12T14:08:43.288 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_unknown": 0, 2020-09-12T14:08:43.288 INFO:tasks.workunit.client.0.smithi027.stdout: "num_osd": 3, 2020-09-12T14:08:43.288 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_sum": { 2020-09-12T14:08:43.288 INFO:tasks.workunit.client.0.smithi027.stdout: "stat_sum": { 2020-09-12T14:08:43.288 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes": 48430, 2020-09-12T14:08:43.289 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects": 101, 2020-09-12T14:08:43.289 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_clones": 1, 2020-09-12T14:08:43.289 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_copies": 303, 2020-09-12T14:08:43.289 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing_on_primary": 0, 2020-09-12T14:08:43.289 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing": 0, 2020-09-12T14:08:43.289 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_degraded": 101, 2020-09-12T14:08:43.289 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_misplaced": 0, 2020-09-12T14:08:43.290 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_unfound": 0, 2020-09-12T14:08:43.290 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_dirty": 101, 2020-09-12T14:08:43.290 INFO:tasks.workunit.client.0.smithi027.stdout: "num_whiteouts": 0, 2020-09-12T14:08:43.290 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read": 0, 2020-09-12T14:08:43.290 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read_kb": 0, 2020-09-12T14:08:43.290 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write": 101, 2020-09-12T14:08:43.290 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write_kb": 101, 2020-09-12T14:08:43.290 INFO:tasks.workunit.client.0.smithi027.stdout: "num_scrub_errors": 0, 2020-09-12T14:08:43.291 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shallow_scrub_errors": 0, 2020-09-12T14:08:43.291 INFO:tasks.workunit.client.0.smithi027.stdout: "num_deep_scrub_errors": 0, 2020-09-12T14:08:43.291 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_recovered": 0, 2020-09-12T14:08:43.291 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_recovered": 0, 2020-09-12T14:08:43.291 INFO:tasks.workunit.client.0.smithi027.stdout: "num_keys_recovered": 0, 2020-09-12T14:08:43.291 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_omap": 0, 2020-09-12T14:08:43.291 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_hit_set_archive": 0, 2020-09-12T14:08:43.291 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_hit_set_archive": 0, 2020-09-12T14:08:43.292 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush": 0, 2020-09-12T14:08:43.292 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_kb": 0, 2020-09-12T14:08:43.292 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict": 0, 2020-09-12T14:08:43.292 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_kb": 0, 2020-09-12T14:08:43.292 INFO:tasks.workunit.client.0.smithi027.stdout: "num_promote": 0, 2020-09-12T14:08:43.292 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_high": 0, 2020-09-12T14:08:43.292 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_low": 0, 2020-09-12T14:08:43.293 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_some": 0, 2020-09-12T14:08:43.293 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_full": 0, 2020-09-12T14:08:43.293 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_pinned": 0, 2020-09-12T14:08:43.293 INFO:tasks.workunit.client.0.smithi027.stdout: "num_legacy_snapsets": 0, 2020-09-12T14:08:43.293 INFO:tasks.workunit.client.0.smithi027.stdout: "num_large_omap_objects": 0, 2020-09-12T14:08:43.293 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_manifest": 0, 2020-09-12T14:08:43.293 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_bytes": 0, 2020-09-12T14:08:43.293 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_keys": 0, 2020-09-12T14:08:43.294 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_repaired": 0 2020-09-12T14:08:43.294 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.294 INFO:tasks.workunit.client.0.smithi027.stdout: "store_stats": { 2020-09-12T14:08:43.294 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 0, 2020-09-12T14:08:43.294 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 0, 2020-09-12T14:08:43.294 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:08:43.294 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 0, 2020-09-12T14:08:43.294 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 0, 2020-09-12T14:08:43.295 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:08:43.295 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:08:43.295 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:08:43.295 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 0, 2020-09-12T14:08:43.295 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 0 2020-09-12T14:08:43.295 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.295 INFO:tasks.workunit.client.0.smithi027.stdout: "log_size": 18, 2020-09-12T14:08:43.295 INFO:tasks.workunit.client.0.smithi027.stdout: "ondisk_log_size": 18, 2020-09-12T14:08:43.296 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 3, 2020-09-12T14:08:43.296 INFO:tasks.workunit.client.0.smithi027.stdout: "acting": 3, 2020-09-12T14:08:43.296 INFO:tasks.workunit.client.0.smithi027.stdout: "num_store_stats": 0 2020-09-12T14:08:43.296 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.296 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_sum": { 2020-09-12T14:08:43.296 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 0, 2020-09-12T14:08:43.296 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 0, 2020-09-12T14:08:43.296 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pgs": 3, 2020-09-12T14:08:43.297 INFO:tasks.workunit.client.0.smithi027.stdout: "num_osds": 3, 2020-09-12T14:08:43.297 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_osds": 3, 2020-09-12T14:08:43.297 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_omap_osds": 3, 2020-09-12T14:08:43.297 INFO:tasks.workunit.client.0.smithi027.stdout: "kb": 314572800, 2020-09-12T14:08:43.297 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used": 19220, 2020-09-12T14:08:43.297 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_data": 2228, 2020-09-12T14:08:43.297 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_omap": 7, 2020-09-12T14:08:43.298 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_meta": 16952, 2020-09-12T14:08:43.298 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_avail": 314553580, 2020-09-12T14:08:43.298 INFO:tasks.workunit.client.0.smithi027.stdout: "statfs": { 2020-09-12T14:08:43.298 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 322122547200, 2020-09-12T14:08:43.298 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 322102865920, 2020-09-12T14:08:43.298 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:08:43.298 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 2281472, 2020-09-12T14:08:43.298 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 1568297, 2020-09-12T14:08:43.299 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:08:43.299 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:08:43.299 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:08:43.299 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 7442, 2020-09-12T14:08:43.299 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 17359598 2020-09-12T14:08:43.299 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.299 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_peers": [], 2020-09-12T14:08:43.299 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_trim_queue_len": 0, 2020-09-12T14:08:43.300 INFO:tasks.workunit.client.0.smithi027.stdout: "num_snap_trimming": 0, 2020-09-12T14:08:43.300 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shards_repaired": 0, 2020-09-12T14:08:43.300 INFO:tasks.workunit.client.0.smithi027.stdout: "op_queue_age_hist": { 2020-09-12T14:08:43.300 INFO:tasks.workunit.client.0.smithi027.stdout: "histogram": [], 2020-09-12T14:08:43.300 INFO:tasks.workunit.client.0.smithi027.stdout: "upper_bound": 1 2020-09-12T14:08:43.300 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.300 INFO:tasks.workunit.client.0.smithi027.stdout: "perf_stat": { 2020-09-12T14:08:43.300 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ms": 0, 2020-09-12T14:08:43.301 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ms": 0, 2020-09-12T14:08:43.301 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ns": 0, 2020-09-12T14:08:43.301 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ns": 0 2020-09-12T14:08:43.301 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.301 INFO:tasks.workunit.client.0.smithi027.stdout: "alerts": [], 2020-09-12T14:08:43.301 INFO:tasks.workunit.client.0.smithi027.stdout: "network_ping_times": [] 2020-09-12T14:08:43.301 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.301 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_sum_by_class": { 2020-09-12T14:08:43.302 INFO:tasks.workunit.client.0.smithi027.stdout: "hdd": { 2020-09-12T14:08:43.302 INFO:tasks.workunit.client.0.smithi027.stdout: "up_from": 0, 2020-09-12T14:08:43.302 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 0, 2020-09-12T14:08:43.302 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pgs": 3, 2020-09-12T14:08:43.302 INFO:tasks.workunit.client.0.smithi027.stdout: "num_osds": 3, 2020-09-12T14:08:43.302 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_osds": 3, 2020-09-12T14:08:43.302 INFO:tasks.workunit.client.0.smithi027.stdout: "num_per_pool_omap_osds": 3, 2020-09-12T14:08:43.303 INFO:tasks.workunit.client.0.smithi027.stdout: "kb": 314572800, 2020-09-12T14:08:43.303 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used": 19220, 2020-09-12T14:08:43.303 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_data": 2228, 2020-09-12T14:08:43.303 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_omap": 7, 2020-09-12T14:08:43.303 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_used_meta": 16952, 2020-09-12T14:08:43.303 INFO:tasks.workunit.client.0.smithi027.stdout: "kb_avail": 314553580, 2020-09-12T14:08:43.303 INFO:tasks.workunit.client.0.smithi027.stdout: "statfs": { 2020-09-12T14:08:43.303 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 322122547200, 2020-09-12T14:08:43.304 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 322102865920, 2020-09-12T14:08:43.304 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:08:43.304 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 2281472, 2020-09-12T14:08:43.304 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 1568297, 2020-09-12T14:08:43.304 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:08:43.304 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:08:43.304 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:08:43.304 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 7442, 2020-09-12T14:08:43.305 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 17359598 2020-09-12T14:08:43.305 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.305 INFO:tasks.workunit.client.0.smithi027.stdout: "hb_peers": [], 2020-09-12T14:08:43.305 INFO:tasks.workunit.client.0.smithi027.stdout: "snap_trim_queue_len": 0, 2020-09-12T14:08:43.305 INFO:tasks.workunit.client.0.smithi027.stdout: "num_snap_trimming": 0, 2020-09-12T14:08:43.305 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shards_repaired": 0, 2020-09-12T14:08:43.305 INFO:tasks.workunit.client.0.smithi027.stdout: "op_queue_age_hist": { 2020-09-12T14:08:43.305 INFO:tasks.workunit.client.0.smithi027.stdout: "histogram": [], 2020-09-12T14:08:43.306 INFO:tasks.workunit.client.0.smithi027.stdout: "upper_bound": 1 2020-09-12T14:08:43.306 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.306 INFO:tasks.workunit.client.0.smithi027.stdout: "perf_stat": { 2020-09-12T14:08:43.306 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ms": 0, 2020-09-12T14:08:43.306 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ms": 0, 2020-09-12T14:08:43.306 INFO:tasks.workunit.client.0.smithi027.stdout: "commit_latency_ns": 0, 2020-09-12T14:08:43.306 INFO:tasks.workunit.client.0.smithi027.stdout: "apply_latency_ns": 0 2020-09-12T14:08:43.307 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.307 INFO:tasks.workunit.client.0.smithi027.stdout: "alerts": [], 2020-09-12T14:08:43.307 INFO:tasks.workunit.client.0.smithi027.stdout: "network_ping_times": [] 2020-09-12T14:08:43.307 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.307 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.307 INFO:tasks.workunit.client.0.smithi027.stdout: "pool_stats": [ 2020-09-12T14:08:43.307 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.307 INFO:tasks.workunit.client.0.smithi027.stdout: "poolid": 1, 2020-09-12T14:08:43.308 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg": 1, 2020-09-12T14:08:43.308 INFO:tasks.workunit.client.0.smithi027.stdout: "stat_sum": { 2020-09-12T14:08:43.308 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes": 48430, 2020-09-12T14:08:43.308 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects": 101, 2020-09-12T14:08:43.308 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_clones": 1, 2020-09-12T14:08:43.308 INFO:tasks.workunit.client.0.smithi027.stdout: "num_object_copies": 303, 2020-09-12T14:08:43.308 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing_on_primary": 0, 2020-09-12T14:08:43.308 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_missing": 0, 2020-09-12T14:08:43.309 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_degraded": 101, 2020-09-12T14:08:43.309 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_misplaced": 0, 2020-09-12T14:08:43.309 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_unfound": 0, 2020-09-12T14:08:43.309 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_dirty": 101, 2020-09-12T14:08:43.309 INFO:tasks.workunit.client.0.smithi027.stdout: "num_whiteouts": 0, 2020-09-12T14:08:43.309 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read": 0, 2020-09-12T14:08:43.309 INFO:tasks.workunit.client.0.smithi027.stdout: "num_read_kb": 0, 2020-09-12T14:08:43.309 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write": 101, 2020-09-12T14:08:43.310 INFO:tasks.workunit.client.0.smithi027.stdout: "num_write_kb": 101, 2020-09-12T14:08:43.310 INFO:tasks.workunit.client.0.smithi027.stdout: "num_scrub_errors": 0, 2020-09-12T14:08:43.310 INFO:tasks.workunit.client.0.smithi027.stdout: "num_shallow_scrub_errors": 0, 2020-09-12T14:08:43.310 INFO:tasks.workunit.client.0.smithi027.stdout: "num_deep_scrub_errors": 0, 2020-09-12T14:08:43.310 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_recovered": 0, 2020-09-12T14:08:43.310 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_recovered": 0, 2020-09-12T14:08:43.310 INFO:tasks.workunit.client.0.smithi027.stdout: "num_keys_recovered": 0, 2020-09-12T14:08:43.311 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_omap": 0, 2020-09-12T14:08:43.311 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_hit_set_archive": 0, 2020-09-12T14:08:43.311 INFO:tasks.workunit.client.0.smithi027.stdout: "num_bytes_hit_set_archive": 0, 2020-09-12T14:08:43.311 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush": 0, 2020-09-12T14:08:43.311 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_kb": 0, 2020-09-12T14:08:43.311 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict": 0, 2020-09-12T14:08:43.311 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_kb": 0, 2020-09-12T14:08:43.311 INFO:tasks.workunit.client.0.smithi027.stdout: "num_promote": 0, 2020-09-12T14:08:43.312 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_high": 0, 2020-09-12T14:08:43.312 INFO:tasks.workunit.client.0.smithi027.stdout: "num_flush_mode_low": 0, 2020-09-12T14:08:43.312 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_some": 0, 2020-09-12T14:08:43.312 INFO:tasks.workunit.client.0.smithi027.stdout: "num_evict_mode_full": 0, 2020-09-12T14:08:43.312 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_pinned": 0, 2020-09-12T14:08:43.312 INFO:tasks.workunit.client.0.smithi027.stdout: "num_legacy_snapsets": 0, 2020-09-12T14:08:43.312 INFO:tasks.workunit.client.0.smithi027.stdout: "num_large_omap_objects": 0, 2020-09-12T14:08:43.312 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_manifest": 0, 2020-09-12T14:08:43.313 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_bytes": 0, 2020-09-12T14:08:43.313 INFO:tasks.workunit.client.0.smithi027.stdout: "num_omap_keys": 0, 2020-09-12T14:08:43.313 INFO:tasks.workunit.client.0.smithi027.stdout: "num_objects_repaired": 0 2020-09-12T14:08:43.313 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.313 INFO:tasks.workunit.client.0.smithi027.stdout: "store_stats": { 2020-09-12T14:08:43.313 INFO:tasks.workunit.client.0.smithi027.stdout: "total": 0, 2020-09-12T14:08:43.313 INFO:tasks.workunit.client.0.smithi027.stdout: "available": 0, 2020-09-12T14:08:43.313 INFO:tasks.workunit.client.0.smithi027.stdout: "internally_reserved": 0, 2020-09-12T14:08:43.314 INFO:tasks.workunit.client.0.smithi027.stdout: "allocated": 1241088, 2020-09-12T14:08:43.314 INFO:tasks.workunit.client.0.smithi027.stdout: "data_stored": 1241088, 2020-09-12T14:08:43.314 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed": 0, 2020-09-12T14:08:43.314 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_allocated": 0, 2020-09-12T14:08:43.314 INFO:tasks.workunit.client.0.smithi027.stdout: "data_compressed_original": 0, 2020-09-12T14:08:43.314 INFO:tasks.workunit.client.0.smithi027.stdout: "omap_allocated": 0, 2020-09-12T14:08:43.314 INFO:tasks.workunit.client.0.smithi027.stdout: "internal_metadata": 0 2020-09-12T14:08:43.315 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.315 INFO:tasks.workunit.client.0.smithi027.stdout: "log_size": 18, 2020-09-12T14:08:43.315 INFO:tasks.workunit.client.0.smithi027.stdout: "ondisk_log_size": 18, 2020-09-12T14:08:43.315 INFO:tasks.workunit.client.0.smithi027.stdout: "up": 3, 2020-09-12T14:08:43.315 INFO:tasks.workunit.client.0.smithi027.stdout: "acting": 3, 2020-09-12T14:08:43.315 INFO:tasks.workunit.client.0.smithi027.stdout: "num_store_stats": 3 2020-09-12T14:08:43.315 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.315 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:08:43.316 INFO:tasks.workunit.client.0.smithi027.stdout: "osd_stats": [ 2020-09-12T14:08:43.316 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.316 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:08:43.316 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 180388626442 2020-09-12T14:08:43.316 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.316 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.316 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:08:43.316 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 42949672982 2020-09-12T14:08:43.317 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.317 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.317 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:08:43.317 INFO:tasks.workunit.client.0.smithi027.stdout: "seq": 193273528330 2020-09-12T14:08:43.317 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.317 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:08:43.317 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_by_state": [ 2020-09-12T14:08:43.317 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.318 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "active+undersized+degraded", 2020-09-12T14:08:43.318 INFO:tasks.workunit.client.0.smithi027.stdout: "num": 1 2020-09-12T14:08:43.318 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.318 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:08:43.318 INFO:tasks.workunit.client.0.smithi027.stdout: "num_pg_by_osd": [ 2020-09-12T14:08:43.318 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.318 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2, 2020-09-12T14:08:43.319 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 0, 2020-09-12T14:08:43.319 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 1, 2020-09-12T14:08:43.319 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T14:08:43.319 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.319 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.319 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 2147483647, 2020-09-12T14:08:43.319 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 0, 2020-09-12T14:08:43.319 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 1, 2020-09-12T14:08:43.320 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T14:08:43.320 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.320 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.320 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 1, 2020-09-12T14:08:43.320 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 0, 2020-09-12T14:08:43.320 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 0, 2020-09-12T14:08:43.320 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T14:08:43.320 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.321 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.321 INFO:tasks.workunit.client.0.smithi027.stdout: "osd": 0, 2020-09-12T14:08:43.321 INFO:tasks.workunit.client.0.smithi027.stdout: "num_primary_pg": 1, 2020-09-12T14:08:43.321 INFO:tasks.workunit.client.0.smithi027.stdout: "num_acting_pg": 1, 2020-09-12T14:08:43.321 INFO:tasks.workunit.client.0.smithi027.stdout: "num_up_not_acting_pg": 0 2020-09-12T14:08:43.321 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.321 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:08:43.321 INFO:tasks.workunit.client.0.smithi027.stdout: "purged_snaps": [ 2020-09-12T14:08:43.322 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:08:43.322 INFO:tasks.workunit.client.0.smithi027.stdout: "pool": 1, 2020-09-12T14:08:43.322 INFO:tasks.workunit.client.0.smithi027.stdout: "purged_snaps": {} 2020-09-12T14:08:43.322 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.322 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:08:43.322 INFO:tasks.workunit.client.0.smithi027.stdout: "servicemap": { 2020-09-12T14:08:43.322 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": 5, 2020-09-12T14:08:43.323 INFO:tasks.workunit.client.0.smithi027.stdout: "modified": "2020-09-12T14:07:11.322124+0000", 2020-09-12T14:08:43.323 INFO:tasks.workunit.client.0.smithi027.stdout: "services": {} 2020-09-12T14:08:43.323 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:08:43.323 INFO:tasks.workunit.client.0.smithi027.stdout: "paxos": { 2020-09-12T14:08:43.323 INFO:tasks.workunit.client.0.smithi027.stdout: "first_committed": 1, 2020-09-12T14:08:43.323 INFO:tasks.workunit.client.0.smithi027.stdout: "last_committed": 514, 2020-09-12T14:08:43.323 INFO:tasks.workunit.client.0.smithi027.stdout: "last_pn": 0, 2020-09-12T14:08:43.323 INFO:tasks.workunit.client.0.smithi027.stdout: "accepted_pn": 0 2020-09-12T14:08:43.324 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:08:43.324 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:08:43.324 INFO:tasks.workunit.client.0.smithi027.stderr:report 3384916126 2020-09-12T14:08:43.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1604: wait_for_clean: return 1 2020-09-12T14:08:43.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:341: TEST_divergent_ec: echo 'delay recovery' 2020-09-12T14:08:43.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:342: TEST_divergent_ec: ceph pg dump pgs 2020-09-12T14:08:43.325 INFO:tasks.workunit.client.0.smithi027.stdout:delay recovery 2020-09-12T14:08:43.560 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:08:43.561 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 101 0 101 0 0 48430 0 0 18 18 active+undersized+degraded 2020-09-12T14:08:05.757019+0000 40'102 47:135 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2020-09-12T14:06:13.000609+0000 0'0 2020-09-12T14:06:13.000609+0000 0 2020-09-12T14:08:43.561 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:08:43.561 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:08:43.562 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:08:43.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:343: TEST_divergent_ec: for i in $non_divergent 2020-09-12T14:08:43.572 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: get_asok_path osd.0 2020-09-12T14:08:43.572 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2020-09-12T14:08:43.572 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2020-09-12T14:08:43.572 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:08:43.572 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:08:43.573 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:08:43.573 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.0.asok 2020-09-12T14:08:43.573 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: CEPH_ARGS= 2020-09-12T14:08:43.573 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.0.asok set_recovery_delay 100000 2020-09-12T14:08:43.680 INFO:tasks.workunit.client.0.smithi027.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2020-09-12T14:08:43.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:343: TEST_divergent_ec: for i in $non_divergent 2020-09-12T14:08:43.689 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: get_asok_path osd.2 2020-09-12T14:08:43.689 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2020-09-12T14:08:43.689 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2020-09-12T14:08:43.689 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:08:43.690 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:08:43.690 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:08:43.690 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.2.asok 2020-09-12T14:08:43.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: CEPH_ARGS= 2020-09-12T14:08:43.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.2.asok set_recovery_delay 100000 2020-09-12T14:08:43.796 INFO:tasks.workunit.client.0.smithi027.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2020-09-12T14:08:43.804 INFO:tasks.workunit.client.0.smithi027.stdout:revive divergent 1 2020-09-12T14:08:43.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:349: TEST_divergent_ec: echo 'revive divergent 1' 2020-09-12T14:08:43.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:350: TEST_divergent_ec: ceph pg dump pgs 2020-09-12T14:08:44.190 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:08:44.190 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 101 0 101 0 0 48430 0 0 18 18 active+undersized+degraded 2020-09-12T14:08:05.757019+0000 40'102 47:135 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2020-09-12T14:06:13.000609+0000 0'0 2020-09-12T14:06:13.000609+0000 0 2020-09-12T14:08:44.191 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:08:44.191 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:08:44.192 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:08:44.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:351: TEST_divergent_ec: ceph osd set noup 2020-09-12T14:08:44.889 INFO:tasks.workunit.client.0.smithi027.stderr:noup is set 2020-09-12T14:08:44.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:352: TEST_divergent_ec: activate_osd td/divergent-priors 1 2020-09-12T14:08:44.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/divergent-priors 2020-09-12T14:08:44.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T14:08:44.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=1 2020-09-12T14:08:44.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T14:08:44.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/divergent-priors/1 2020-09-12T14:08:44.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T14:08:44.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:08:44.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:08:44.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:08:44.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2020-09-12T14:08:44.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2020-09-12T14:08:44.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T14:08:44.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T14:08:44.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T14:08:44.902 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T14:08:44.902 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:08:44.902 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:08:44.902 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:08:44.903 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:08:44.903 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:08:44.903 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:08:44.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:08:44.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:08:44.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T14:08:44.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T14:08:44.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:08:44.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:08:44.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:08:44.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T14:08:44.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T14:08:44.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/divergent-priors/1 2020-09-12T14:08:44.905 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T14:08:44.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.1 2020-09-12T14:08:44.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 1 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:08:44.906 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/divergent-priors/1/whoami 2020-09-12T14:08:44.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 1 = 1 ']' 2020-09-12T14:08:44.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T14:08:44.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T14:08:44.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T14:08:44.929 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:08:44.929+0000 7f14b5e70ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:08:44.931 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:08:44.930+0000 7f14b5e70ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:08:44.932 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:08:44.932+0000 7f14b5e70ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:08:45.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:877: activate_osd: return 0 2020-09-12T14:08:45.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:353: TEST_divergent_ec: sleep 5 2020-09-12T14:08:45.448 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:08:45.447+0000 7f14b5e70ec0 -1 Falling back to public interface 2020-09-12T14:08:46.369 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:08:46.368+0000 7f14b5e70ec0 -1 osd.1 30 log_to_monitors {default=true} 2020-09-12T14:08:50.311 INFO:tasks.workunit.client.0.smithi027.stdout:delay recovery divergent 2020-09-12T14:08:50.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:355: TEST_divergent_ec: echo 'delay recovery divergent' 2020-09-12T14:08:50.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:356: TEST_divergent_ec: ceph pg dump pgs 2020-09-12T14:08:50.704 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:08:50.705 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 101 0 101 0 0 48430 0 0 18 18 active+undersized+degraded 2020-09-12T14:08:05.757019+0000 40'102 47:136 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2020-09-12T14:06:13.000609+0000 0'0 2020-09-12T14:06:13.000609+0000 0 2020-09-12T14:08:50.705 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:08:50.705 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:08:50.706 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:08:50.716 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:357: TEST_divergent_ec: get_asok_path osd.1 2020-09-12T14:08:50.716 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2020-09-12T14:08:50.716 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2020-09-12T14:08:50.716 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:08:50.716 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:08:50.717 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:08:50.717 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20815/ceph-osd.1.asok 2020-09-12T14:08:50.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:357: TEST_divergent_ec: CEPH_ARGS= 2020-09-12T14:08:50.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:357: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.20815/ceph-osd.1.asok set_recovery_delay 100000 2020-09-12T14:08:50.824 INFO:tasks.workunit.client.0.smithi027.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2020-09-12T14:08:50.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:359: TEST_divergent_ec: ceph osd unset noup 2020-09-12T14:08:51.557 INFO:tasks.workunit.client.0.smithi027.stderr:noup is unset 2020-09-12T14:08:51.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:361: TEST_divergent_ec: wait_for_osd up 0 2020-09-12T14:08:51.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:08:51.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T14:08:51.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:08:51.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:08:51.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:08:51.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:08:51.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:08:51.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:08:51.568 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:08:52.042 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 42 up_thru 45 down_at 41 last_clean_interval [36,40) [v2:127.0.0.1:6802/63138,v1:127.0.0.1:6803/63138] [v2:127.0.0.1:6804/63138,v1:127.0.0.1:6805/63138] exists,up 3d802381-3749-43a0-ac4c-e7fa2730ba5f 2020-09-12T14:08:52.043 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:08:52.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:08:52.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:08:52.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:08:52.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:362: TEST_divergent_ec: wait_for_osd up 1 2020-09-12T14:08:52.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:08:52.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:08:52.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:08:52.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:08:52.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:08:52.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:08:52.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:08:52.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:08:52.450 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 51 up_thru 51 down_at 32 last_clean_interval [10,30) [v2:127.0.0.1:6818/65483,v1:127.0.0.1:6819/65483] [v2:127.0.0.1:6820/65483,v1:127.0.0.1:6821/65483] exists,up 270594ae-5b9c-4b0c-bd1d-3fddd2a993d0 2020-09-12T14:08:52.451 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:08:52.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:08:52.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:08:52.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:08:52.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:363: TEST_divergent_ec: wait_for_osd up 2 2020-09-12T14:08:52.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:08:52.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T14:08:52.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:08:52.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:08:52.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:08:52.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:08:52.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:08:52.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:08:52.862 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up in weight 1 up_from 45 up_thru 0 down_at 44 last_clean_interval [37,40) [v2:127.0.0.1:6810/63787,v1:127.0.0.1:6811/63787] [v2:127.0.0.1:6812/63787,v1:127.0.0.1:6813/63787] exists,up ed2b134c-2770-4bad-8557-6c53d9f7b711 2020-09-12T14:08:52.862 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:08:52.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:08:52.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:08:52.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:365: TEST_divergent_ec: ceph pg dump pgs 2020-09-12T14:08:53.261 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:08:53.262 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 101 0 101 0 0 48430 0 0 18 18 active+undersized+degraded 2020-09-12T14:08:05.757019+0000 40'102 47:136 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2020-09-12T14:06:13.000609+0000 0'0 2020-09-12T14:06:13.000609+0000 0 2020-09-12T14:08:53.262 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:08:53.262 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:08:53.263 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:08:53.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:366: TEST_divergent_ec: echo 'wait for peering' 2020-09-12T14:08:53.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:367: TEST_divergent_ec: ceph pg dump pgs 2020-09-12T14:08:53.273 INFO:tasks.workunit.client.0.smithi027.stdout:wait for peering 2020-09-12T14:08:53.667 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:08:53.667 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 101 0 101 0 0 48430 0 0 18 18 active+undersized+degraded 2020-09-12T14:08:05.757019+0000 40'102 47:136 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2020-09-12T14:06:13.000609+0000 0'0 2020-09-12T14:06:13.000609+0000 0 2020-09-12T14:08:53.668 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:08:53.668 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:08:53.669 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:08:53.678 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:368: TEST_divergent_ec: rados -p test put foo /etc/fstab 2020-09-12T14:09:01.822 INFO:tasks.workunit.client.0.smithi027.stdout:killing divergent 1 2020-09-12T14:09:01.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:370: TEST_divergent_ec: echo 'killing divergent 1' 2020-09-12T14:09:01.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:371: TEST_divergent_ec: ceph pg dump pgs 2020-09-12T14:09:02.218 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:09:02.219 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 101 4 4 0 0 48430 0 0 12 12 active+recovering+degraded+wait 2020-09-12T14:08:52.483060+0000 40'102 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:06:13.000609+0000 0'0 2020-09-12T14:06:13.000609+0000 0 2020-09-12T14:09:02.219 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:09:02.219 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:09:02.220 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:09:02.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:372: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd.1 2020-09-12T14:09:02.230 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:09:02.230 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:09:02.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:09:02.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:09:02.231 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:09:02.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:09:02.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:374: TEST_divergent_ec: echo 'reviving divergent 1' 2020-09-12T14:09:02.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:375: TEST_divergent_ec: ceph pg dump pgs 2020-09-12T14:09:02.336 INFO:tasks.workunit.client.0.smithi027.stdout:reviving divergent 1 2020-09-12T14:09:02.722 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:09:02.722 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 101 4 4 0 0 48430 0 0 12 12 active+recovering+degraded+wait 2020-09-12T14:08:52.483060+0000 40'102 52:127 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:06:13.000609+0000 0'0 2020-09-12T14:06:13.000609+0000 0 2020-09-12T14:09:02.722 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:09:02.722 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:09:02.724 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:09:02.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:376: TEST_divergent_ec: activate_osd td/divergent-priors 1 2020-09-12T14:09:02.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/divergent-priors 2020-09-12T14:09:02.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T14:09:02.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=1 2020-09-12T14:09:02.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T14:09:02.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/divergent-priors/1 2020-09-12T14:09:02.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2020-09-12T14:09:02.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:09:02.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:09:02.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:09:02.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2020-09-12T14:09:02.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2020-09-12T14:09:02.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T14:09:02.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T14:09:02.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2020-09-12T14:09:02.738 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T14:09:02.739 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:09:02.739 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:09:02.739 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:09:02.739 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:09:02.740 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:09:02.740 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:09:02.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' 2020-09-12T14:09:02.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:09:02.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2020-09-12T14:09:02.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2020-09-12T14:09:02.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:09:02.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:09:02.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:09:02.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T14:09:02.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T14:09:02.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/divergent-priors/1 2020-09-12T14:09:02.744 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T14:09:02.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.1 2020-09-12T14:09:02.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 1 --fsid=7e90bad3-7973-4a8c-be65-6326ab931fe8 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20815/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:09:02.745 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/divergent-priors/1/whoami 2020-09-12T14:09:02.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 1 = 1 ']' 2020-09-12T14:09:02.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T14:09:02.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T14:09:02.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T14:09:02.765 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:02.763+0000 7fd59cde6ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:02.766 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:02.765+0000 7fd59cde6ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:02.768 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:02.766+0000 7fd59cde6ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:03.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: wait_for_osd up 1 2020-09-12T14:09:03.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:09:03.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:09:03.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:09:03.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:09:03.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:09:03.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:09:03.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:09:03.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:09:03.144 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:09:03.283 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:03.281+0000 7fd59cde6ec0 -1 Falling back to public interface 2020-09-12T14:09:03.566 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 51 up_thru 51 down_at 32 last_clean_interval [10,30) [v2:127.0.0.1:6818/65483,v1:127.0.0.1:6819/65483] [v2:127.0.0.1:6820/65483,v1:127.0.0.1:6821/65483] exists,up 270594ae-5b9c-4b0c-bd1d-3fddd2a993d0 2020-09-12T14:09:03.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:09:03.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:09:03.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:09:03.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:378: TEST_divergent_ec: sleep 20 2020-09-12T14:09:04.325 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:04.317+0000 7fd59cde6ec0 -1 osd.1 52 log_to_monitors {default=true} 2020-09-12T14:09:23.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:380: TEST_divergent_ec: echo 'allowing recovery' 2020-09-12T14:09:23.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:381: TEST_divergent_ec: ceph pg dump pgs 2020-09-12T14:09:23.568 INFO:tasks.workunit.client.0.smithi027.stdout:allowing recovery 2020-09-12T14:09:23.974 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:09:23.975 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 102 0 0 0 0 48913 0 0 13 13 active+clean 2020-09-12T14:09:06.240538+0000 52'103 56:152 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:06:13.000609+0000 0'0 2020-09-12T14:06:13.000609+0000 0 2020-09-12T14:09:23.975 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:09:23.975 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:09:23.976 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:09:23.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:383: TEST_divergent_ec: for i in $osds 2020-09-12T14:09:23.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:385: TEST_divergent_ec: ceph tell osd.0 debug kick_recovery_wq 0 2020-09-12T14:09:24.123 INFO:tasks.workunit.client.0.smithi027.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2020-09-12T14:09:24.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:383: TEST_divergent_ec: for i in $osds 2020-09-12T14:09:24.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:385: TEST_divergent_ec: ceph tell osd.1 debug kick_recovery_wq 0 2020-09-12T14:09:24.269 INFO:tasks.workunit.client.0.smithi027.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2020-09-12T14:09:24.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:383: TEST_divergent_ec: for i in $osds 2020-09-12T14:09:24.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:385: TEST_divergent_ec: ceph tell osd.2 debug kick_recovery_wq 0 2020-09-12T14:09:24.416 INFO:tasks.workunit.client.0.smithi027.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2020-09-12T14:09:24.424 INFO:tasks.workunit.client.0.smithi027.stdout:reading divergent objects 2020-09-12T14:09:24.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:388: TEST_divergent_ec: echo 'reading divergent objects' 2020-09-12T14:09:24.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:389: TEST_divergent_ec: ceph pg dump pgs 2020-09-12T14:09:24.816 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:09:24.817 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 102 0 0 0 0 48913 0 0 13 13 active+clean 2020-09-12T14:09:06.240538+0000 52'103 56:152 [1,0,2] 1 [1,0,2] 1 0'0 2020-09-12T14:06:13.000609+0000 0'0 2020-09-12T14:06:13.000609+0000 0 2020-09-12T14:09:24.817 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:09:24.817 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:09:24.818 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:09:24.828 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: expr 2 + 2 2020-09-12T14:09:24.829 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: seq 1 4 2020-09-12T14:09:24.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2020-09-12T14:09:24.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:392: TEST_divergent_ec: rados -p test get existing_1 td/divergent-priors/existing 2020-09-12T14:09:24.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2020-09-12T14:09:24.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:392: TEST_divergent_ec: rados -p test get existing_2 td/divergent-priors/existing 2020-09-12T14:09:24.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2020-09-12T14:09:24.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:392: TEST_divergent_ec: rados -p test get existing_3 td/divergent-priors/existing 2020-09-12T14:09:24.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2020-09-12T14:09:24.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:392: TEST_divergent_ec: rados -p test get existing_4 td/divergent-priors/existing 2020-09-12T14:09:25.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:394: TEST_divergent_ec: rm -f td/divergent-priors/existing 2020-09-12T14:09:25.004 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:396: TEST_divergent_ec: find td/divergent-priors -name '*osd*log' 2020-09-12T14:09:25.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:396: TEST_divergent_ec: grep _merge_object_divergent_entries td/divergent-priors/osd.2.log td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log 2020-09-12T14:09:25.009 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 20 _merge_object_divergent_entries: merging hoid 1:7a80c0d5:::existing_99:1 entries: 30'103 (29'99) clone 1:7a80c0d5:::existing_99:1 by unknown.0.0:0 2020-09-12T14:06:48.944678+0000 0 snaps [1] ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 2020-09-12T14:09:25.009 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 20 _merge_object_divergent_entries: keeping 30'103 (29'99) clone 1:7a80c0d5:::existing_99:1 by unknown.0.0:0 2020-09-12T14:06:48.944678+0000 0 snaps [1] ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 2020-09-12T14:09:25.009 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2020-09-12T14:09:25.009 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:1 prior_version: 29'99 first_divergent_update: 30'103 last_divergent_update: 30'103 2020-09-12T14:09:25.009 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:1 has no more recent entries in log 2020-09-12T14:09:25.010 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:1 prior_version or op type indicates creation, deleting 2020-09-12T14:09:25.010 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 20 _merge_object_divergent_entries: merging hoid 1:7a80c0d5:::existing_99:head entries: 30'104 (29'99) modify 1:7a80c0d5:::existing_99:head by client.4555.0:1 2020-09-12T14:06:56.652282+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0 2020-09-12T14:09:25.010 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 20 _merge_object_divergent_entries: keeping 30'104 (29'99) modify 1:7a80c0d5:::existing_99:head by client.4555.0:1 2020-09-12T14:06:56.652282+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0 2020-09-12T14:09:25.010 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2020-09-12T14:09:25.010 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head prior_version: 29'99 first_divergent_update: 30'104 last_divergent_update: 30'104 2020-09-12T14:09:25.010 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head has no more recent entries in log 2020-09-12T14:09:25.010 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head must be rolled back or recovered, attempting to rollback 2020-09-12T14:09:25.011 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head olog_can_rollback_to: 30'104 2020-09-12T14:09:25.011 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head cannot rollback 30'104 (29'99) modify 1:7a80c0d5:::existing_99:head by client.4555.0:1 2020-09-12T14:06:56.652282+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0 2020-09-12T14:09:25.011 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head cannot roll back, removing and adding to missing 2020-09-12T14:09:25.011 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 20 _merge_object_divergent_entries: merging hoid 1:c156033e:::existing_100:head entries: 29'101 (29'100) modify 1:c156033e:::existing_100:head by client.4546.0:1 2020-09-12T14:06:55.431605+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0,29'102 (29'101) modify 1:c156033e:::existing_100:head by client.4552.0:1 2020-09-12T14:06:56.424181+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0 2020-09-12T14:09:25.011 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 20 _merge_object_divergent_entries: keeping 29'101 (29'100) modify 1:c156033e:::existing_100:head by client.4546.0:1 2020-09-12T14:06:55.431605+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0 2020-09-12T14:09:25.012 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 20 _merge_object_divergent_entries: keeping 29'102 (29'101) modify 1:c156033e:::existing_100:head by client.4552.0:1 2020-09-12T14:06:56.424181+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0 2020-09-12T14:09:25.012 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2020-09-12T14:09:25.012 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head prior_version: 29'100 first_divergent_update: 29'101 last_divergent_update: 29'102 2020-09-12T14:09:25.012 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head has no more recent entries in log 2020-09-12T14:09:25.012 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head must be rolled back or recovered, attempting to rollback 2020-09-12T14:09:25.012 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head olog_can_rollback_to: 30'104 2020-09-12T14:09:25.012 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot rollback 29'102 (29'101) modify 1:c156033e:::existing_100:head by client.4552.0:1 2020-09-12T14:06:56.424181+0000 0 ObjectCleanRegions clean_offsets: [483~18446744073709551132], clean_omap: 1, new_object: 0 2020-09-12T14:09:25.013 INFO:tasks.workunit.client.0.smithi027.stdout:td/divergent-priors/osd.1.log:2020-09-12T14:08:52.306+0000 7f1493572700 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot roll back, removing and adding to missing 2020-09-12T14:09:25.013 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:405: TEST_divergent_ec: find td/divergent-priors -name '*osd*log' 2020-09-12T14:09:25.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:405: TEST_divergent_ec: grep -q '_merge_object_divergent_entries.*rolled back' td/divergent-priors/osd.2.log td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log 2020-09-12T14:09:25.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:410: TEST_divergent_ec: echo success 2020-09-12T14:09:25.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:412: TEST_divergent_ec: delete_pool test 2020-09-12T14:09:25.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: delete_pool: local poolname=test 2020-09-12T14:09:25.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2020-09-12T14:09:25.017 INFO:tasks.workunit.client.0.smithi027.stdout:success 2020-09-12T14:09:25.576 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test' does not exist 2020-09-12T14:09:25.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:413: TEST_divergent_ec: kill_daemons td/divergent-priors 2020-09-12T14:09:25.586 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:09:25.586 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:09:25.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:09:25.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:09:25.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:09:25.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:09:25.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2020-09-12T14:09:25.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2020-09-12T14:09:25.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:09:25.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2020-09-12T14:09:25.896 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:09:25.896 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:09:25.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:09:25.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:09:25.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:09:25.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:09:25.901 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:09:25.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:09:25.902 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:09:25.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:09:25.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:09:25.903 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:09:25.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:09:25.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:09:25.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:09:25.905 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:09:25.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:09:25.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:09:25.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2020-09-12T14:09:25.919 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:09:25.919 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:09:25.919 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:09:25.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.20815 2020-09-12T14:09:25.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:09:25.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:09:25.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2151: main: code=0 2020-09-12T14:09:25.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2155: main: teardown td/divergent-priors 0 2020-09-12T14:09:25.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2020-09-12T14:09:25.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2020-09-12T14:09:25.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2020-09-12T14:09:25.922 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:09:25.922 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:09:25.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:09:25.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:09:25.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:09:25.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:09:25.923 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:09:25.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:09:25.924 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:09:25.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:09:25.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:09:25.925 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:09:25.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:09:25.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:09:25.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:09:25.928 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:09:25.928 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:09:25.929 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2020-09-12T14:09:25.929 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2020-09-12T14:09:25.930 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:09:25.930 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:09:25.930 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20815 2020-09-12T14:09:25.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.20815 2020-09-12T14:09:25.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:09:25.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:09:25.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2156: main: return 0 2020-09-12T14:09:25.932 INFO:teuthology.orchestra.run:Running command with timeout 3600 2020-09-12T14:09:25.932 INFO:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2020-09-12T14:09:25.973 INFO:tasks.workunit:Running workunit osd/ec-error-rollforward.sh... 2020-09-12T14:09:25.973 INFO:teuthology.orchestra.run.smithi027:workunit test osd/ec-error-rollforward.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f346cc73d15a568d3bec1c99a58953de76f57b85 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh 2020-09-12T14:09:26.034 INFO:tasks.workunit.client.0.smithi027.stderr:stty: 'standard input': Inappropriate ioctl for device 2020-09-12T14:09:26.041 INFO:tasks.workunit.client.0.smithi027.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2020-09-12T14:09:26.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2020-09-12T14:09:26.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2020-09-12T14:09:26.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2145: main: export PYTHONWARNINGS=ignore 2020-09-12T14:09:26.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2145: main: PYTHONWARNINGS=ignore 2020-09-12T14:09:26.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2146: main: export CEPH_CONF=/dev/null 2020-09-12T14:09:26.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2146: main: CEPH_CONF=/dev/null 2020-09-12T14:09:26.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2147: main: unset CEPH_ARGS 2020-09-12T14:09:26.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2149: main: local code 2020-09-12T14:09:26.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2150: main: run td/ec-error-rollforward 2020-09-12T14:09:26.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:6: run: local dir=td/ec-error-rollforward 2020-09-12T14:09:26.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:7: run: shift 2020-09-12T14:09:26.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:10: run: export CEPH_MON=127.0.0.1:7132 2020-09-12T14:09:26.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:10: run: CEPH_MON=127.0.0.1:7132 2020-09-12T14:09:26.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:11: run: export CEPH_ARGS 2020-09-12T14:09:26.043 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:12: run: uuidgen 2020-09-12T14:09:26.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:12: run: CEPH_ARGS+='--fsid=2f745fc9-5230-4cd7-8578-f6285ceda0e1 --auth-supported=none ' 2020-09-12T14:09:26.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:13: run: CEPH_ARGS+='--mon-host=127.0.0.1:7132 ' 2020-09-12T14:09:26.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:14: run: export margin=10 2020-09-12T14:09:26.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:14: run: margin=10 2020-09-12T14:09:26.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:15: run: export objects=200 2020-09-12T14:09:26.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:15: run: objects=200 2020-09-12T14:09:26.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:16: run: export poolname=test 2020-09-12T14:09:26.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:16: run: poolname=test 2020-09-12T14:09:26.046 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:18: run: set 2020-09-12T14:09:26.046 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:18: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2020-09-12T14:09:26.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:18: run: local funcs=TEST_ec_error_rollforward 2020-09-12T14:09:26.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:19: run: for func in $funcs 2020-09-12T14:09:26.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:20: run: setup td/ec-error-rollforward 2020-09-12T14:09:26.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/ec-error-rollforward 2020-09-12T14:09:26.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/ec-error-rollforward 2020-09-12T14:09:26.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/ec-error-rollforward 2020-09-12T14:09:26.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:09:26.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/ec-error-rollforward KILL 2020-09-12T14:09:26.047 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:09:26.048 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:09:26.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:09:26.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:09:26.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:09:26.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:09:26.049 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:09:26.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:09:26.049 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:09:26.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:09:26.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:09:26.050 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:09:26.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:09:26.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:09:26.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:09:26.051 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:09:26.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:09:26.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:09:26.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/ec-error-rollforward 2020-09-12T14:09:26.052 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:09:26.052 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:09:26.052 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.66788 2020-09-12T14:09:26.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.66788 2020-09-12T14:09:26.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:09:26.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:09:26.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/ec-error-rollforward 2020-09-12T14:09:26.054 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2020-09-12T14:09:26.054 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:09:26.054 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.66788 2020-09-12T14:09:26.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.66788 2020-09-12T14:09:26.055 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2020-09-12T14:09:26.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2020-09-12T14:09:26.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2020-09-12T14:09:26.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2020-09-12T14:09:26.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/ec-error-rollforward 1' TERM HUP INT 2020-09-12T14:09:26.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:21: run: TEST_ec_error_rollforward td/ec-error-rollforward 2020-09-12T14:09:26.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:27: TEST_ec_error_rollforward: local dir=td/ec-error-rollforward 2020-09-12T14:09:26.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:28: TEST_ec_error_rollforward: run_mon td/ec-error-rollforward a 2020-09-12T14:09:26.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/ec-error-rollforward 2020-09-12T14:09:26.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2020-09-12T14:09:26.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2020-09-12T14:09:26.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2020-09-12T14:09:26.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/ec-error-rollforward/a 2020-09-12T14:09:26.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/ec-error-rollforward/a --run-dir=td/ec-error-rollforward 2020-09-12T14:09:26.295 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2020-09-12T14:09:26.295 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:09:26.296 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:09:26.296 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:09:26.296 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:09:26.296 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.66788 2020-09-12T14:09:26.296 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.66788/$cluster-$name.asok' 2020-09-12T14:09:26.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ec-error-rollforward/a '--log-file=td/ec-error-rollforward/$name.log' '--admin-socket=/tmp/ceph-asok.66788/$cluster-$name.asok' --mon-cluster-log-file=td/ec-error-rollforward/log --run-dir=td/ec-error-rollforward '--pid-file=td/ec-error-rollforward/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 2020-09-12T14:09:26.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: cat 2020-09-12T14:09:26.492 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a fsid 2020-09-12T14:09:26.492 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:09:26.492 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:09:26.492 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=fsid 2020-09-12T14:09:26.492 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .fsid 2020-09-12T14:09:26.493 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:09:26.493 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:09:26.493 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:09:26.493 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:09:26.493 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:09:26.493 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.66788 2020-09-12T14:09:26.493 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.66788/ceph-mon.a.asok 2020-09-12T14:09:26.494 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:09:26.494 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.66788/ceph-mon.a.asok config get fsid 2020-09-12T14:09:26.595 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a mon_host 2020-09-12T14:09:26.596 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:09:26.596 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:09:26.596 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=mon_host 2020-09-12T14:09:26.596 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .mon_host 2020-09-12T14:09:26.596 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:09:26.596 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:09:26.596 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:09:26.597 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:09:26.597 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:09:26.598 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.66788 2020-09-12T14:09:26.598 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.66788/ceph-mon.a.asok 2020-09-12T14:09:26.598 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:09:26.598 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.66788/ceph-mon.a.asok config get mon_host 2020-09-12T14:09:26.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:29: TEST_ec_error_rollforward: run_mgr td/ec-error-rollforward x 2020-09-12T14:09:26.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: local dir=td/ec-error-rollforward 2020-09-12T14:09:26.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: shift 2020-09-12T14:09:26.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: local id=x 2020-09-12T14:09:26.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: shift 2020-09-12T14:09:26.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: local data=td/ec-error-rollforward/x 2020-09-12T14:09:26.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:558: run_mgr: ceph config set mgr mgr/devicehealth/enable_monitoring off --force 2020-09-12T14:09:27.612 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: get_asok_path 2020-09-12T14:09:27.613 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:09:27.613 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:09:27.613 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:09:27.613 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:09:27.613 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.66788 2020-09-12T14:09:27.613 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.66788/$cluster-$name.asok' 2020-09-12T14:09:27.614 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:09:27.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ec-error-rollforward/x '--log-file=td/ec-error-rollforward/$name.log' '--admin-socket=/tmp/ceph-asok.66788/$cluster-$name.asok' --run-dir=td/ec-error-rollforward '--pid-file=td/ec-error-rollforward/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:09:27.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:30: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 0 2020-09-12T14:09:27.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/ec-error-rollforward 2020-09-12T14:09:27.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:09:27.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=0 2020-09-12T14:09:27.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:09:27.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/ec-error-rollforward/0 2020-09-12T14:09:27.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=2f745fc9-5230-4cd7-8578-f6285ceda0e1 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2020-09-12T14:09:27.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:09:27.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:09:27.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:09:27.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/0' 2020-09-12T14:09:27.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/0/journal' 2020-09-12T14:09:27.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:09:27.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:09:27.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2020-09-12T14:09:27.638 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:09:27.638 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:09:27.638 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:09:27.639 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:09:27.639 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:09:27.639 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.66788 2020-09-12T14:09:27.639 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.66788/$cluster-$name.asok' 2020-09-12T14:09:27.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66788/$cluster-$name.asok' 2020-09-12T14:09:27.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:09:27.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:09:27.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:09:27.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2020-09-12T14:09:27.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2020-09-12T14:09:27.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:09:27.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:09:27.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:09:27.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:09:27.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:09:27.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/ec-error-rollforward/0 2020-09-12T14:09:27.641 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:09:27.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=25044766-7bfd-4186-9aab-ebf5c17639b9 2020-09-12T14:09:27.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd0 25044766-7bfd-4186-9aab-ebf5c17639b9' 2020-09-12T14:09:27.642 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:09:27.642 INFO:tasks.workunit.client.0.smithi027.stdout:add osd0 25044766-7bfd-4186-9aab-ebf5c17639b9 2020-09-12T14:09:27.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQCX1lxfn7vHJhAAO2uMSTYyzwk3rsvWFWNEdA== 2020-09-12T14:09:27.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQCX1lxfn7vHJhAAO2uMSTYyzwk3rsvWFWNEdA=="}' 2020-09-12T14:09:27.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 25044766-7bfd-4186-9aab-ebf5c17639b9 -i td/ec-error-rollforward/0/new.json 2020-09-12T14:09:28.042 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:09:28.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/ec-error-rollforward/0/new.json 2020-09-12T14:09:28.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 0 --fsid=2f745fc9-5230-4cd7-8578-f6285ceda0e1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/0 --osd-journal=td/ec-error-rollforward/0/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.66788/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQCX1lxfn7vHJhAAO2uMSTYyzwk3rsvWFWNEdA== --osd-uuid 25044766-7bfd-4186-9aab-ebf5c17639b9 2020-09-12T14:09:28.075 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:28.074+0000 7f8192899ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:28.077 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:28.076+0000 7f8192899ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:28.077 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:28.077+0000 7f8192899ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:28.078 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:28.078+0000 7f8192899ec0 -1 bluestore(td/ec-error-rollforward/0/block) _read_bdev_label failed to open td/ec-error-rollforward/0/block: (2) No such file or directory 2020-09-12T14:09:28.078 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:28.078+0000 7f8192899ec0 -1 bluestore(td/ec-error-rollforward/0/block) _read_bdev_label failed to open td/ec-error-rollforward/0/block: (2) No such file or directory 2020-09-12T14:09:28.079 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:28.078+0000 7f8192899ec0 -1 bluestore(td/ec-error-rollforward/0/block) _read_bdev_label failed to open td/ec-error-rollforward/0/block: (2) No such file or directory 2020-09-12T14:09:28.113 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:28.113+0000 7f8192899ec0 -1 bluestore(td/ec-error-rollforward/0) _read_fsid unparsable uuid 2020-09-12T14:09:32.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/ec-error-rollforward/0/keyring 2020-09-12T14:09:32.550 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:09:32.550 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd0 key to auth repository 2020-09-12T14:09:32.550 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/ec-error-rollforward/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:09:32.551 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd0 key to auth repository 2020-09-12T14:09:33.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.0 2020-09-12T14:09:33.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 0 --fsid=2f745fc9-5230-4cd7-8578-f6285ceda0e1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/0 --osd-journal=td/ec-error-rollforward/0/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.66788/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:09:33.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:09:33.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:09:33.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:09:33.097 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.0 2020-09-12T14:09:33.115 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:33.113+0000 7fe8cc9fbec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:33.145 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:33.144+0000 7fe8cc9fbec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:33.147 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:33.145+0000 7fe8cc9fbec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:33.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 0 2020-09-12T14:09:33.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:09:33.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T14:09:33.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:09:33.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:09:33.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:09:33.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:09:33.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:09:33.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:09:33.501 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:09:33.666 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:33.664+0000 7fe8cc9fbec0 -1 Falling back to public interface 2020-09-12T14:09:33.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:09:34.481 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:34.479+0000 7fe8cc9fbec0 -1 osd.0 0 log_to_monitors {default=true} 2020-09-12T14:09:34.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:09:34.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:09:34.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:09:34.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:09:34.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:09:34.898 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:09:35.309 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/67646,v1:127.0.0.1:6803/67646] [v2:127.0.0.1:6804/67646,v1:127.0.0.1:6805/67646] exists,up 25044766-7bfd-4186-9aab-ebf5c17639b9 2020-09-12T14:09:35.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:09:35.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:09:35.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:09:35.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:31: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 1 2020-09-12T14:09:35.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/ec-error-rollforward 2020-09-12T14:09:35.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:09:35.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=1 2020-09-12T14:09:35.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:09:35.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/ec-error-rollforward/1 2020-09-12T14:09:35.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=2f745fc9-5230-4cd7-8578-f6285ceda0e1 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2020-09-12T14:09:35.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:09:35.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:09:35.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:09:35.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/1' 2020-09-12T14:09:35.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/1/journal' 2020-09-12T14:09:35.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:09:35.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:09:35.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2020-09-12T14:09:35.312 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:09:35.312 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:09:35.312 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:09:35.313 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:09:35.313 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:09:35.313 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.66788 2020-09-12T14:09:35.313 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.66788/$cluster-$name.asok' 2020-09-12T14:09:35.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66788/$cluster-$name.asok' 2020-09-12T14:09:35.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:09:35.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:09:35.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:09:35.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2020-09-12T14:09:35.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2020-09-12T14:09:35.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:09:35.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:09:35.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:09:35.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:09:35.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:09:35.315 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/ec-error-rollforward/1 2020-09-12T14:09:35.315 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:09:35.315 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=d7f36afb-f37c-47ea-8370-db2d50988e4c 2020-09-12T14:09:35.316 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd1 d7f36afb-f37c-47ea-8370-db2d50988e4c' 2020-09-12T14:09:35.316 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:09:35.316 INFO:tasks.workunit.client.0.smithi027.stdout:add osd1 d7f36afb-f37c-47ea-8370-db2d50988e4c 2020-09-12T14:09:35.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQCf1lxfPsk/ExAAdVofjH5+bm4ot0mZhZ7hFg== 2020-09-12T14:09:35.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQCf1lxfPsk/ExAAdVofjH5+bm4ot0mZhZ7hFg=="}' 2020-09-12T14:09:35.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new d7f36afb-f37c-47ea-8370-db2d50988e4c -i td/ec-error-rollforward/1/new.json 2020-09-12T14:09:35.879 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:09:35.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/ec-error-rollforward/1/new.json 2020-09-12T14:09:35.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 1 --fsid=2f745fc9-5230-4cd7-8578-f6285ceda0e1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/1 --osd-journal=td/ec-error-rollforward/1/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.66788/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQCf1lxfPsk/ExAAdVofjH5+bm4ot0mZhZ7hFg== --osd-uuid d7f36afb-f37c-47ea-8370-db2d50988e4c 2020-09-12T14:09:35.913 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:35.912+0000 7f2969935ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:35.936 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:35.935+0000 7f2969935ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:35.937 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:35.936+0000 7f2969935ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:35.938 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:35.936+0000 7f2969935ec0 -1 bluestore(td/ec-error-rollforward/1/block) _read_bdev_label failed to open td/ec-error-rollforward/1/block: (2) No such file or directory 2020-09-12T14:09:35.938 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:35.936+0000 7f2969935ec0 -1 bluestore(td/ec-error-rollforward/1/block) _read_bdev_label failed to open td/ec-error-rollforward/1/block: (2) No such file or directory 2020-09-12T14:09:35.938 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:35.936+0000 7f2969935ec0 -1 bluestore(td/ec-error-rollforward/1/block) _read_bdev_label failed to open td/ec-error-rollforward/1/block: (2) No such file or directory 2020-09-12T14:09:36.003 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:36.001+0000 7f2969935ec0 -1 bluestore(td/ec-error-rollforward/1) _read_fsid unparsable uuid 2020-09-12T14:09:40.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/ec-error-rollforward/1/keyring 2020-09-12T14:09:40.261 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:09:40.262 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd1 key to auth repository 2020-09-12T14:09:40.262 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/ec-error-rollforward/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:09:40.262 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd1 key to auth repository 2020-09-12T14:09:40.797 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T14:09:40.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.1 2020-09-12T14:09:40.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 1 --fsid=2f745fc9-5230-4cd7-8578-f6285ceda0e1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/1 --osd-journal=td/ec-error-rollforward/1/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.66788/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:09:40.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:09:40.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:09:40.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:09:40.817 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:40.815+0000 7fe46ca4eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:40.818 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:40.817+0000 7fe46ca4eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:40.820 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:40.818+0000 7fe46ca4eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:41.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 1 2020-09-12T14:09:41.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:09:41.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:09:41.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:09:41.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:09:41.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:09:41.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:09:41.196 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:09:41.196 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:09:41.196 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:09:41.341 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:41.340+0000 7fe46ca4eec0 -1 Falling back to public interface 2020-09-12T14:09:41.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:09:42.178 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:42.176+0000 7fe46ca4eec0 -1 osd.1 0 log_to_monitors {default=true} 2020-09-12T14:09:42.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:09:42.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:09:42.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:09:42.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:09:42.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:09:42.634 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:09:43.041 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/68640,v1:127.0.0.1:6811/68640] [v2:127.0.0.1:6812/68640,v1:127.0.0.1:6813/68640] exists,up d7f36afb-f37c-47ea-8370-db2d50988e4c 2020-09-12T14:09:43.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:09:43.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:09:43.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:09:43.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:32: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 2 2020-09-12T14:09:43.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/ec-error-rollforward 2020-09-12T14:09:43.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:09:43.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=2 2020-09-12T14:09:43.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:09:43.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/ec-error-rollforward/2 2020-09-12T14:09:43.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=2f745fc9-5230-4cd7-8578-f6285ceda0e1 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2020-09-12T14:09:43.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:09:43.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:09:43.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:09:43.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/2' 2020-09-12T14:09:43.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/2/journal' 2020-09-12T14:09:43.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:09:43.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:09:43.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2020-09-12T14:09:43.046 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:09:43.046 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:09:43.046 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:09:43.046 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:09:43.047 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:09:43.047 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.66788 2020-09-12T14:09:43.047 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.66788/$cluster-$name.asok' 2020-09-12T14:09:43.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66788/$cluster-$name.asok' 2020-09-12T14:09:43.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:09:43.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:09:43.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:09:43.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2020-09-12T14:09:43.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2020-09-12T14:09:43.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:09:43.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:09:43.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:09:43.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:09:43.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:09:43.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/ec-error-rollforward/2 2020-09-12T14:09:43.049 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:09:43.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=c557dc56-eee7-4fde-9943-89f06f29ec68 2020-09-12T14:09:43.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd2 c557dc56-eee7-4fde-9943-89f06f29ec68' 2020-09-12T14:09:43.049 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:09:43.049 INFO:tasks.workunit.client.0.smithi027.stdout:add osd2 c557dc56-eee7-4fde-9943-89f06f29ec68 2020-09-12T14:09:43.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQCn1lxfTzxzAxAAbLF5hjjOxzu9aGC+zHhP7w== 2020-09-12T14:09:43.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQCn1lxfTzxzAxAAbLF5hjjOxzu9aGC+zHhP7w=="}' 2020-09-12T14:09:43.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new c557dc56-eee7-4fde-9943-89f06f29ec68 -i td/ec-error-rollforward/2/new.json 2020-09-12T14:09:43.601 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:09:43.612 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/ec-error-rollforward/2/new.json 2020-09-12T14:09:43.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 2 --fsid=2f745fc9-5230-4cd7-8578-f6285ceda0e1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/2 --osd-journal=td/ec-error-rollforward/2/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.66788/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQCn1lxfTzxzAxAAbLF5hjjOxzu9aGC+zHhP7w== --osd-uuid c557dc56-eee7-4fde-9943-89f06f29ec68 2020-09-12T14:09:43.634 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:43.633+0000 7f895a5d3ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:43.636 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:43.634+0000 7f895a5d3ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:43.637 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:43.636+0000 7f895a5d3ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:43.638 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:43.636+0000 7f895a5d3ec0 -1 bluestore(td/ec-error-rollforward/2/block) _read_bdev_label failed to open td/ec-error-rollforward/2/block: (2) No such file or directory 2020-09-12T14:09:43.638 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:43.636+0000 7f895a5d3ec0 -1 bluestore(td/ec-error-rollforward/2/block) _read_bdev_label failed to open td/ec-error-rollforward/2/block: (2) No such file or directory 2020-09-12T14:09:43.638 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:43.636+0000 7f895a5d3ec0 -1 bluestore(td/ec-error-rollforward/2/block) _read_bdev_label failed to open td/ec-error-rollforward/2/block: (2) No such file or directory 2020-09-12T14:09:43.670 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:43.669+0000 7f895a5d3ec0 -1 bluestore(td/ec-error-rollforward/2) _read_fsid unparsable uuid 2020-09-12T14:09:47.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/ec-error-rollforward/2/keyring 2020-09-12T14:09:47.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:09:47.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd2 key to auth repository 2020-09-12T14:09:47.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/ec-error-rollforward/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:09:47.911 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd2 key to auth repository 2020-09-12T14:09:48.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.2 2020-09-12T14:09:48.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:09:48.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:09:48.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:09:48.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 2 --fsid=2f745fc9-5230-4cd7-8578-f6285ceda0e1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/2 --osd-journal=td/ec-error-rollforward/2/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.66788/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:09:48.462 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.2 2020-09-12T14:09:48.481 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:48.480+0000 7f860087fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:48.483 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:48.481+0000 7f860087fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:48.484 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:48.483+0000 7f860087fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:48.859 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 2 2020-09-12T14:09:48.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:09:48.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T14:09:48.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:09:48.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:09:48.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:09:48.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:09:48.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:09:48.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:09:48.861 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:09:49.005 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:49.004+0000 7f860087fec0 -1 Falling back to public interface 2020-09-12T14:09:49.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:09:49.870 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:49.870+0000 7f860087fec0 -1 osd.2 0 log_to_monitors {default=true} 2020-09-12T14:09:50.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:09:50.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:09:50.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:09:50.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:09:50.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:09:50.279 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:09:50.742 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/69634,v1:127.0.0.1:6819/69634] [v2:127.0.0.1:6820/69634,v1:127.0.0.1:6821/69634] exists,up c557dc56-eee7-4fde-9943-89f06f29ec68 2020-09-12T14:09:50.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:09:50.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:09:50.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:09:50.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:33: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 3 2020-09-12T14:09:50.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/ec-error-rollforward 2020-09-12T14:09:50.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:09:50.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=3 2020-09-12T14:09:50.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:09:50.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/ec-error-rollforward/3 2020-09-12T14:09:50.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=2f745fc9-5230-4cd7-8578-f6285ceda0e1 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2020-09-12T14:09:50.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:09:50.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:09:50.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:09:50.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/3' 2020-09-12T14:09:50.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/3/journal' 2020-09-12T14:09:50.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:09:50.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:09:50.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2020-09-12T14:09:50.745 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:09:50.746 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:09:50.746 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:09:50.746 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:09:50.746 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:09:50.746 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.66788 2020-09-12T14:09:50.746 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.66788/$cluster-$name.asok' 2020-09-12T14:09:50.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66788/$cluster-$name.asok' 2020-09-12T14:09:50.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:09:50.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:09:50.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:09:50.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2020-09-12T14:09:50.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2020-09-12T14:09:50.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:09:50.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:09:50.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:09:50.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:09:50.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:09:50.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/ec-error-rollforward/3 2020-09-12T14:09:50.750 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:09:50.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=170daba2-128a-4c63-92bd-7130d431837b 2020-09-12T14:09:50.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd3 170daba2-128a-4c63-92bd-7130d431837b' 2020-09-12T14:09:50.750 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:09:50.750 INFO:tasks.workunit.client.0.smithi027.stdout:add osd3 170daba2-128a-4c63-92bd-7130d431837b 2020-09-12T14:09:50.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQCu1lxfAHA2LRAAAzVDrhlF1DnNKCBUBb4h6A== 2020-09-12T14:09:50.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQCu1lxfAHA2LRAAAzVDrhlF1DnNKCBUBb4h6A=="}' 2020-09-12T14:09:50.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 170daba2-128a-4c63-92bd-7130d431837b -i td/ec-error-rollforward/3/new.json 2020-09-12T14:09:51.300 INFO:tasks.workunit.client.0.smithi027.stdout:3 2020-09-12T14:09:51.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/ec-error-rollforward/3/new.json 2020-09-12T14:09:51.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 3 --fsid=2f745fc9-5230-4cd7-8578-f6285ceda0e1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/3 --osd-journal=td/ec-error-rollforward/3/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.66788/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQCu1lxfAHA2LRAAAzVDrhlF1DnNKCBUBb4h6A== --osd-uuid 170daba2-128a-4c63-92bd-7130d431837b 2020-09-12T14:09:51.333 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:51.333+0000 7f68c7116ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:51.335 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:51.334+0000 7f68c7116ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:51.336 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:51.336+0000 7f68c7116ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:51.337 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:51.336+0000 7f68c7116ec0 -1 bluestore(td/ec-error-rollforward/3/block) _read_bdev_label failed to open td/ec-error-rollforward/3/block: (2) No such file or directory 2020-09-12T14:09:51.337 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:51.336+0000 7f68c7116ec0 -1 bluestore(td/ec-error-rollforward/3/block) _read_bdev_label failed to open td/ec-error-rollforward/3/block: (2) No such file or directory 2020-09-12T14:09:51.337 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:51.336+0000 7f68c7116ec0 -1 bluestore(td/ec-error-rollforward/3/block) _read_bdev_label failed to open td/ec-error-rollforward/3/block: (2) No such file or directory 2020-09-12T14:09:51.386 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:51.385+0000 7f68c7116ec0 -1 bluestore(td/ec-error-rollforward/3) _read_fsid unparsable uuid 2020-09-12T14:09:55.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/ec-error-rollforward/3/keyring 2020-09-12T14:09:55.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:09:55.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd3 key to auth repository 2020-09-12T14:09:55.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/ec-error-rollforward/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:09:55.689 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd3 key to auth repository 2020-09-12T14:09:56.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.3 2020-09-12T14:09:56.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 3 --fsid=2f745fc9-5230-4cd7-8578-f6285ceda0e1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/3 --osd-journal=td/ec-error-rollforward/3/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.66788/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:09:56.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:09:56.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:09:56.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:09:56.233 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.3 2020-09-12T14:09:56.252 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:56.251+0000 7fb3f299dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:56.293 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:56.293+0000 7fb3f299dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:56.295 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:56.294+0000 7fb3f299dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:09:56.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 3 2020-09-12T14:09:56.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:09:56.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=3 2020-09-12T14:09:56.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:09:56.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:09:56.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:09:56.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:09:56.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:09:56.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.3 up' 2020-09-12T14:09:56.631 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:09:56.811 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:56.810+0000 7fb3f299dec0 -1 Falling back to public interface 2020-09-12T14:09:57.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:09:57.721 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:09:57.720+0000 7fb3f299dec0 -1 osd.3 0 log_to_monitors {default=true} 2020-09-12T14:09:58.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:09:58.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:09:58.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:09:58.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:09:58.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.3 up' 2020-09-12T14:09:58.045 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:09:58.470 INFO:tasks.workunit.client.0.smithi027.stdout:osd.3 up in weight 1 up_from 20 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/70637,v1:127.0.0.1:6827/70637] [v2:127.0.0.1:6828/70637,v1:127.0.0.1:6829/70637] exists,up 170daba2-128a-4c63-92bd-7130d431837b 2020-09-12T14:09:58.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:09:58.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:09:58.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:09:58.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:35: TEST_ec_error_rollforward: ceph osd erasure-code-profile set ec-profile m=2 k=2 crush-failure-domain=osd 2020-09-12T14:09:59.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:36: TEST_ec_error_rollforward: ceph osd pool create ec 1 1 erasure ec-profile 2020-09-12T14:09:59.910 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'ec' already exists 2020-09-12T14:09:59.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:38: TEST_ec_error_rollforward: rados -p ec put foo /etc/passwd 2020-09-12T14:10:01.462 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:40: TEST_ec_error_rollforward: cat td/ec-error-rollforward/osd.2.pid 2020-09-12T14:10:01.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:40: TEST_ec_error_rollforward: kill -STOP 69634 2020-09-12T14:10:01.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:43: TEST_ec_error_rollforward: pids=71091 2020-09-12T14:10:01.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:44: TEST_ec_error_rollforward: sleep 1 2020-09-12T14:10:01.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:42: TEST_ec_error_rollforward: rados -p ec rm foo 2020-09-12T14:10:02.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:46: TEST_ec_error_rollforward: pids+=' 71112' 2020-09-12T14:10:02.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:45: TEST_ec_error_rollforward: rados -p ec rm a 2020-09-12T14:10:02.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:48: TEST_ec_error_rollforward: pids+=' 71113' 2020-09-12T14:10:02.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:47: TEST_ec_error_rollforward: rados -p ec rm b 2020-09-12T14:10:02.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:50: TEST_ec_error_rollforward: pids+=' 71114' 2020-09-12T14:10:02.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:51: TEST_ec_error_rollforward: sleep 1 2020-09-12T14:10:02.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:49: TEST_ec_error_rollforward: rados -p ec rm c 2020-09-12T14:10:03.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:54: TEST_ec_error_rollforward: kill_daemons td/ec-error-rollforward KILL osd 2020-09-12T14:10:03.460 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:10:03.460 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:10:03.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:10:03.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:10:03.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:10:03.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:10:03.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:55: TEST_ec_error_rollforward: kill 71091 71112 71113 71114 2020-09-12T14:10:03.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:56: TEST_ec_error_rollforward: wait 2020-09-12T14:10:03.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:58: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 0 2020-09-12T14:10:03.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/ec-error-rollforward 2020-09-12T14:10:03.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T14:10:03.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=0 2020-09-12T14:10:03.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T14:10:03.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/ec-error-rollforward/0 2020-09-12T14:10:03.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=2f745fc9-5230-4cd7-8578-f6285ceda0e1 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2020-09-12T14:10:03.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:10:03.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:10:03.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:10:03.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/0' 2020-09-12T14:10:03.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/0/journal' 2020-09-12T14:10:03.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T14:10:03.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T14:10:03.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2020-09-12T14:10:03.577 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T14:10:03.577 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:10:03.577 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:10:03.577 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:10:03.577 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:10:03.577 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.66788 2020-09-12T14:10:03.578 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.66788/$cluster-$name.asok' 2020-09-12T14:10:03.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66788/$cluster-$name.asok' 2020-09-12T14:10:03.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:10:03.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2020-09-12T14:10:03.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2020-09-12T14:10:03.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:10:03.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:10:03.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:10:03.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T14:10:03.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T14:10:03.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/ec-error-rollforward/0 2020-09-12T14:10:03.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.0 2020-09-12T14:10:03.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 0 --fsid=2f745fc9-5230-4cd7-8578-f6285ceda0e1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/0 --osd-journal=td/ec-error-rollforward/0/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.66788/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:10:03.580 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/ec-error-rollforward/0/whoami 2020-09-12T14:10:03.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 0 = 0 ']' 2020-09-12T14:10:03.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T14:10:03.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T14:10:03.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T14:10:03.581 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.0 2020-09-12T14:10:03.598 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:03.597+0000 7fd147348ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:03.600 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:03.600+0000 7fd147348ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:03.601 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:03.601+0000 7fd147348ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:03.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: wait_for_osd up 0 2020-09-12T14:10:03.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:10:03.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T14:10:03.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:10:03.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:10:03.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:10:03.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:10:03.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:10:03.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:10:03.993 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:10:04.118 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:04.117+0000 7fd147348ec0 -1 Falling back to public interface 2020-09-12T14:10:04.393 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/67646,v1:127.0.0.1:6803/67646] [v2:127.0.0.1:6804/67646,v1:127.0.0.1:6805/67646] exists,up 25044766-7bfd-4186-9aab-ebf5c17639b9 2020-09-12T14:10:04.394 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T14:10:04.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:10:04.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:10:04.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:10:04.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:59: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 1 2020-09-12T14:10:04.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/ec-error-rollforward 2020-09-12T14:10:04.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T14:10:04.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=1 2020-09-12T14:10:04.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T14:10:04.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/ec-error-rollforward/1 2020-09-12T14:10:04.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=2f745fc9-5230-4cd7-8578-f6285ceda0e1 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2020-09-12T14:10:04.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:10:04.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:10:04.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:10:04.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/1' 2020-09-12T14:10:04.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/1/journal' 2020-09-12T14:10:04.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T14:10:04.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T14:10:04.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2020-09-12T14:10:04.397 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T14:10:04.397 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:10:04.397 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:10:04.397 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:10:04.397 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:10:04.398 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.66788 2020-09-12T14:10:04.398 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.66788/$cluster-$name.asok' 2020-09-12T14:10:04.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66788/$cluster-$name.asok' 2020-09-12T14:10:04.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:10:04.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2020-09-12T14:10:04.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2020-09-12T14:10:04.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:10:04.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:10:04.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:10:04.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T14:10:04.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T14:10:04.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/ec-error-rollforward/1 2020-09-12T14:10:04.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.1 2020-09-12T14:10:04.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 1 --fsid=2f745fc9-5230-4cd7-8578-f6285ceda0e1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/1 --osd-journal=td/ec-error-rollforward/1/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.66788/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:10:04.400 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/ec-error-rollforward/1/whoami 2020-09-12T14:10:04.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 1 = 1 ']' 2020-09-12T14:10:04.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T14:10:04.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T14:10:04.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T14:10:04.423 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:04.422+0000 7fadc5dc1ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:04.425 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:04.424+0000 7fadc5dc1ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:04.426 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:04.425+0000 7fadc5dc1ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:04.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: wait_for_osd up 1 2020-09-12T14:10:04.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:10:04.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:10:04.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:10:04.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:10:04.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:10:04.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:10:04.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:10:04.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:10:04.809 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:10:04.946 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:04.945+0000 7fadc5dc1ec0 -1 Falling back to public interface 2020-09-12T14:10:05.212 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 10 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/68640,v1:127.0.0.1:6811/68640] [v2:127.0.0.1:6812/68640,v1:127.0.0.1:6813/68640] exists,up d7f36afb-f37c-47ea-8370-db2d50988e4c 2020-09-12T14:10:05.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:10:05.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:10:05.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:10:05.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:60: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 2 2020-09-12T14:10:05.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/ec-error-rollforward 2020-09-12T14:10:05.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T14:10:05.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=2 2020-09-12T14:10:05.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T14:10:05.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/ec-error-rollforward/2 2020-09-12T14:10:05.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=2f745fc9-5230-4cd7-8578-f6285ceda0e1 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2020-09-12T14:10:05.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:10:05.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:10:05.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:10:05.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/2' 2020-09-12T14:10:05.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/2/journal' 2020-09-12T14:10:05.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T14:10:05.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T14:10:05.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2020-09-12T14:10:05.215 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T14:10:05.215 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:10:05.215 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:10:05.216 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:10:05.216 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:10:05.216 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.66788 2020-09-12T14:10:05.216 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.66788/$cluster-$name.asok' 2020-09-12T14:10:05.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66788/$cluster-$name.asok' 2020-09-12T14:10:05.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:10:05.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2020-09-12T14:10:05.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2020-09-12T14:10:05.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:10:05.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:10:05.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:10:05.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T14:10:05.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T14:10:05.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/ec-error-rollforward/2 2020-09-12T14:10:05.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.2 2020-09-12T14:10:05.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 2 --fsid=2f745fc9-5230-4cd7-8578-f6285ceda0e1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/2 --osd-journal=td/ec-error-rollforward/2/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.66788/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:10:05.218 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/ec-error-rollforward/2/whoami 2020-09-12T14:10:05.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 2 = 2 ']' 2020-09-12T14:10:05.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T14:10:05.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T14:10:05.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T14:10:05.219 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.2 2020-09-12T14:10:05.232 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:05.232+0000 7f10ddf60ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:05.234 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:05.234+0000 7f10ddf60ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:05.236 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:05.235+0000 7f10ddf60ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:05.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: wait_for_osd up 2 2020-09-12T14:10:05.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:10:05.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T14:10:05.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:10:05.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:10:05.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:10:05.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:10:05.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:10:05.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:10:05.711 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:10:05.717 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:05.716+0000 7fd147348ec0 -1 osd.0 25 log_to_monitors {default=true} 2020-09-12T14:10:05.754 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:05.753+0000 7f10ddf60ec0 -1 Falling back to public interface 2020-09-12T14:10:06.187 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/69634,v1:127.0.0.1:6819/69634] [v2:127.0.0.1:6820/69634,v1:127.0.0.1:6821/69634] exists,up c557dc56-eee7-4fde-9943-89f06f29ec68 2020-09-12T14:10:06.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:10:06.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:10:06.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:10:06.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:61: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 3 2020-09-12T14:10:06.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/ec-error-rollforward 2020-09-12T14:10:06.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T14:10:06.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=3 2020-09-12T14:10:06.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T14:10:06.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/ec-error-rollforward/3 2020-09-12T14:10:06.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=2f745fc9-5230-4cd7-8578-f6285ceda0e1 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2020-09-12T14:10:06.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:10:06.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:10:06.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:10:06.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/3' 2020-09-12T14:10:06.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/3/journal' 2020-09-12T14:10:06.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T14:10:06.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T14:10:06.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2020-09-12T14:10:06.191 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T14:10:06.191 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:10:06.192 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:10:06.192 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:10:06.192 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:10:06.192 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.66788 2020-09-12T14:10:06.192 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.66788/$cluster-$name.asok' 2020-09-12T14:10:06.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66788/$cluster-$name.asok' 2020-09-12T14:10:06.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:10:06.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2020-09-12T14:10:06.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2020-09-12T14:10:06.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:10:06.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:10:06.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:10:06.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T14:10:06.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T14:10:06.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/ec-error-rollforward/3 2020-09-12T14:10:06.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.3 2020-09-12T14:10:06.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 3 --fsid=2f745fc9-5230-4cd7-8578-f6285ceda0e1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/3 --osd-journal=td/ec-error-rollforward/3/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.66788/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:10:06.194 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/ec-error-rollforward/3/whoami 2020-09-12T14:10:06.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 3 = 3 ']' 2020-09-12T14:10:06.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T14:10:06.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T14:10:06.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T14:10:06.195 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.3 2020-09-12T14:10:06.209 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:06.208+0000 7fa5a727cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:06.210 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:06.210+0000 7fa5a727cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:06.211 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:06.211+0000 7fa5a727cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:06.732 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:06.731+0000 7fa5a727cec0 -1 Falling back to public interface 2020-09-12T14:10:06.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: wait_for_osd up 3 2020-09-12T14:10:06.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:10:06.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=3 2020-09-12T14:10:06.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:10:06.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:10:06.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:10:06.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:10:06.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:10:06.740 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:10:06.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.3 up' 2020-09-12T14:10:07.044 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:07.043+0000 7fadc5dc1ec0 -1 osd.1 25 log_to_monitors {default=true} 2020-09-12T14:10:07.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:10:08.097 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:08.097+0000 7f10ddf60ec0 -1 osd.2 24 log_to_monitors {default=true} 2020-09-12T14:10:08.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:10:08.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:10:08.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:10:08.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:10:08.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.3 up' 2020-09-12T14:10:08.217 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:10:08.782 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:10:09.176 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:09.175+0000 7fa5a727cec0 -1 osd.3 25 log_to_monitors {default=true} 2020-09-12T14:10:09.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:10:09.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:10:09.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 2 2020-09-12T14:10:09.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:10:09.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.3 up' 2020-09-12T14:10:09.784 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:10:10.205 INFO:tasks.workunit.client.0.smithi027.stdout:osd.3 up in weight 1 up_from 37 up_thru 0 down_at 28 last_clean_interval [20,25) [v2:127.0.0.1:6826/72146,v1:127.0.0.1:6827/72146] [v2:127.0.0.1:6828/72146,v1:127.0.0.1:6829/72146] exists,up 170daba2-128a-4c63-92bd-7130d431837b 2020-09-12T14:10:10.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:10:10.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:10:10.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:10:10.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:63: TEST_ec_error_rollforward: wait_for_clean 2020-09-12T14:10:10.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:10:10.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:10:10.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:10:10.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:10:10.208 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 90 .1 2020-09-12T14:10:10.208 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:10:10.208 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:10:10.208 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:10:10.208 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:10:10.208 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:10:10.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:10:10.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:10:10.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:10:10.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:10:10.300 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:10:10.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:10:10.746 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:10:10.747 INFO:tasks.workunit.client.0.smithi027.stderr:2 2020-09-12T14:10:10.747 INFO:tasks.workunit.client.0.smithi027.stderr:3' 2020-09-12T14:10:10.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:10:10.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:10:10.748 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:10:10.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=115964116996 2020-09-12T14:10:10.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 115964116996 2020-09-12T14:10:10.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-115964116996' 2020-09-12T14:10:10.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:10:10.887 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:10:11.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=128849018883 2020-09-12T14:10:11.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 128849018883 2020-09-12T14:10:11.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-115964116996 1-128849018883' 2020-09-12T14:10:11.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:10:11.025 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:10:11.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=146028888067 2020-09-12T14:10:11.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 146028888067 2020-09-12T14:10:11.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-115964116996 1-128849018883 2-146028888067' 2020-09-12T14:10:11.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:10:11.163 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2020-09-12T14:10:11.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=158913789955 2020-09-12T14:10:11.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 158913789955 2020-09-12T14:10:11.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-115964116996 1-128849018883 2-146028888067 3-158913789955' 2020-09-12T14:10:11.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:10:11.301 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-115964116996 2020-09-12T14:10:11.301 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:10:11.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:10:11.303 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-115964116996 2020-09-12T14:10:11.303 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:10:11.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=115964116996 2020-09-12T14:10:11.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 115964116996' 2020-09-12T14:10:11.304 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 115964116996 2020-09-12T14:10:11.304 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:10:11.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 115964116996 -lt 115964116996 2020-09-12T14:10:11.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:10:11.724 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-128849018883 2020-09-12T14:10:11.724 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:10:11.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:10:11.726 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-128849018883 2020-09-12T14:10:11.727 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:10:11.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=128849018883 2020-09-12T14:10:11.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 128849018883' 2020-09-12T14:10:11.728 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 128849018883 2020-09-12T14:10:11.728 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:10:12.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 128849018883 -lt 128849018883 2020-09-12T14:10:12.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:10:12.126 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-146028888067 2020-09-12T14:10:12.126 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:10:12.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:10:12.128 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-146028888067 2020-09-12T14:10:12.128 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:10:12.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=146028888067 2020-09-12T14:10:12.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 146028888067' 2020-09-12T14:10:12.130 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 146028888067 2020-09-12T14:10:12.130 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:10:12.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 146028888067 -lt 146028888067 2020-09-12T14:10:12.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:10:12.567 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 3-158913789955 2020-09-12T14:10:12.568 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:10:12.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=3 2020-09-12T14:10:12.569 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 3-158913789955 2020-09-12T14:10:12.570 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:10:12.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=158913789955 2020-09-12T14:10:12.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.3 seq 158913789955' 2020-09-12T14:10:12.571 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.3 seq 158913789955 2020-09-12T14:10:12.571 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 3 2020-09-12T14:10:12.966 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 158913789955 -lt 158913789955 2020-09-12T14:10:12.967 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:10:12.967 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:10:12.968 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:10:13.496 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 1 == 0 2020-09-12T14:10:13.496 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:10:13.497 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:10:13.497 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:10:13.497 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:10:13.497 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:10:13.497 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:10:13.497 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:10:13.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:10:13.889 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:10:13.889 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:10:13.889 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:10:14.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 1 2020-09-12T14:10:14.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T14:10:14.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T14:10:14.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:22: run: teardown td/ec-error-rollforward 2020-09-12T14:10:14.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/ec-error-rollforward 2020-09-12T14:10:14.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:10:14.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/ec-error-rollforward KILL 2020-09-12T14:10:14.410 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:10:14.411 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:10:14.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:10:14.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:10:14.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:10:14.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:10:14.519 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:10:14.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:10:14.520 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:10:14.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:10:14.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:10:14.521 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:10:14.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:10:14.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:10:14.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:10:14.524 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:10:14.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:10:14.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:10:14.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/ec-error-rollforward 2020-09-12T14:10:14.530 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:10:14.530 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:10:14.530 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.66788 2020-09-12T14:10:14.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.66788 2020-09-12T14:10:14.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:10:14.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:10:14.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2151: main: code=0 2020-09-12T14:10:14.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2155: main: teardown td/ec-error-rollforward 0 2020-09-12T14:10:14.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/ec-error-rollforward 2020-09-12T14:10:14.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2020-09-12T14:10:14.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/ec-error-rollforward KILL 2020-09-12T14:10:14.533 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:10:14.533 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:10:14.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:10:14.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:10:14.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:10:14.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:10:14.534 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:10:14.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:10:14.535 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:10:14.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:10:14.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:10:14.536 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:10:14.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:10:14.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:10:14.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:10:14.538 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:10:14.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:10:14.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2020-09-12T14:10:14.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/ec-error-rollforward 2020-09-12T14:10:14.540 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:10:14.540 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:10:14.540 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.66788 2020-09-12T14:10:14.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.66788 2020-09-12T14:10:14.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:10:14.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:10:14.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2156: main: return 0 2020-09-12T14:10:14.542 INFO:teuthology.orchestra.run:Running command with timeout 3600 2020-09-12T14:10:14.543 INFO:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2020-09-12T14:10:14.612 INFO:tasks.workunit:Running workunit osd/osd-backfill-prio.sh... 2020-09-12T14:10:14.612 INFO:teuthology.orchestra.run.smithi027:workunit test osd/osd-backfill-prio.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f346cc73d15a568d3bec1c99a58953de76f57b85 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh 2020-09-12T14:10:14.673 INFO:tasks.workunit.client.0.smithi027.stderr:stty: 'standard input': Inappropriate ioctl for device 2020-09-12T14:10:14.681 INFO:tasks.workunit.client.0.smithi027.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2020-09-12T14:10:14.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2020-09-12T14:10:14.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2020-09-12T14:10:14.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2145: main: export PYTHONWARNINGS=ignore 2020-09-12T14:10:14.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2145: main: PYTHONWARNINGS=ignore 2020-09-12T14:10:14.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2146: main: export CEPH_CONF=/dev/null 2020-09-12T14:10:14.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2146: main: CEPH_CONF=/dev/null 2020-09-12T14:10:14.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2147: main: unset CEPH_ARGS 2020-09-12T14:10:14.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2149: main: local code 2020-09-12T14:10:14.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2150: main: run td/osd-backfill-prio 2020-09-12T14:10:14.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:21: run: local dir=td/osd-backfill-prio 2020-09-12T14:10:14.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:22: run: shift 2020-09-12T14:10:14.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:25: run: export CEPH_MON=127.0.0.1:7114 2020-09-12T14:10:14.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:25: run: CEPH_MON=127.0.0.1:7114 2020-09-12T14:10:14.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:26: run: export CEPH_ARGS 2020-09-12T14:10:14.684 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:27: run: uuidgen 2020-09-12T14:10:14.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:27: run: CEPH_ARGS+='--fsid=576afb9f-26d0-41c8-a49c-302fc5ab0911 --auth-supported=none ' 2020-09-12T14:10:14.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7114 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:10:14.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:29: run: CEPH_ARGS+='--osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 ' 2020-09-12T14:10:14.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:30: run: export objects=50 2020-09-12T14:10:14.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:30: run: objects=50 2020-09-12T14:10:14.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:31: run: export poolprefix=test 2020-09-12T14:10:14.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:31: run: poolprefix=test 2020-09-12T14:10:14.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:32: run: export FORCE_PRIO=254 2020-09-12T14:10:14.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:32: run: FORCE_PRIO=254 2020-09-12T14:10:14.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:33: run: export DEGRADED_PRIO=150 2020-09-12T14:10:14.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:33: run: DEGRADED_PRIO=150 2020-09-12T14:10:14.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:34: run: export NORMAL_PRIO=110 2020-09-12T14:10:14.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:34: run: NORMAL_PRIO=110 2020-09-12T14:10:14.688 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:36: run: set 2020-09-12T14:10:14.688 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:36: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2020-09-12T14:10:14.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:36: run: local 'funcs=TEST_backfill_pool_priority 2020-09-12T14:10:14.689 INFO:tasks.workunit.client.0.smithi027.stderr:TEST_backfill_priority' 2020-09-12T14:10:14.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:37: run: for func in $funcs 2020-09-12T14:10:14.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:38: run: setup td/osd-backfill-prio 2020-09-12T14:10:14.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-backfill-prio 2020-09-12T14:10:14.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-backfill-prio 2020-09-12T14:10:14.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-backfill-prio 2020-09-12T14:10:14.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:10:14.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-backfill-prio KILL 2020-09-12T14:10:14.690 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:10:14.690 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:10:14.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:10:14.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:10:14.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:10:14.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:10:14.690 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:10:14.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:10:14.692 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:10:14.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:10:14.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:10:14.692 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:10:14.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:10:14.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:10:14.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:10:14.693 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:10:14.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:10:14.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:10:14.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-backfill-prio 2020-09-12T14:10:14.694 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:10:14.694 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:10:14.694 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:10:14.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.73258 2020-09-12T14:10:14.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:10:14.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:10:14.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-backfill-prio 2020-09-12T14:10:14.695 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2020-09-12T14:10:14.696 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:10:14.696 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:10:14.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.73258 2020-09-12T14:10:14.696 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2020-09-12T14:10:14.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2020-09-12T14:10:14.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2020-09-12T14:10:14.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2020-09-12T14:10:14.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-backfill-prio 1' TERM HUP INT 2020-09-12T14:10:14.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:39: run: TEST_backfill_pool_priority td/osd-backfill-prio 2020-09-12T14:10:14.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:326: TEST_backfill_pool_priority: local dir=td/osd-backfill-prio 2020-09-12T14:10:14.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:327: TEST_backfill_pool_priority: local pools=3 2020-09-12T14:10:14.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:328: TEST_backfill_pool_priority: local OSDS=2 2020-09-12T14:10:14.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:330: TEST_backfill_pool_priority: run_mon td/osd-backfill-prio a 2020-09-12T14:10:14.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-backfill-prio 2020-09-12T14:10:14.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2020-09-12T14:10:14.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2020-09-12T14:10:14.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2020-09-12T14:10:14.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-backfill-prio/a 2020-09-12T14:10:14.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-backfill-prio/a --run-dir=td/osd-backfill-prio 2020-09-12T14:10:14.941 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2020-09-12T14:10:14.941 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:10:14.942 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:10:14.942 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:10:14.942 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:10:14.942 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:10:14.942 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.73258/$cluster-$name.asok' 2020-09-12T14:10:14.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-backfill-prio/a '--log-file=td/osd-backfill-prio/$name.log' '--admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' --mon-cluster-log-file=td/osd-backfill-prio/log --run-dir=td/osd-backfill-prio '--pid-file=td/osd-backfill-prio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 2020-09-12T14:10:15.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: cat 2020-09-12T14:10:15.149 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a fsid 2020-09-12T14:10:15.150 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:10:15.150 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:10:15.150 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=fsid 2020-09-12T14:10:15.150 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .fsid 2020-09-12T14:10:15.150 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:10:15.150 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:10:15.151 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:10:15.151 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:10:15.151 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:10:15.152 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:10:15.152 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.73258/ceph-mon.a.asok 2020-09-12T14:10:15.152 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:10:15.152 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.73258/ceph-mon.a.asok config get fsid 2020-09-12T14:10:15.255 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a mon_host 2020-09-12T14:10:15.255 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:10:15.255 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:10:15.256 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=mon_host 2020-09-12T14:10:15.256 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .mon_host 2020-09-12T14:10:15.256 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:10:15.256 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:10:15.256 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:10:15.256 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:10:15.257 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:10:15.257 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:10:15.257 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.73258/ceph-mon.a.asok 2020-09-12T14:10:15.257 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:10:15.258 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.73258/ceph-mon.a.asok config get mon_host 2020-09-12T14:10:15.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:331: TEST_backfill_pool_priority: run_mgr td/osd-backfill-prio x 2020-09-12T14:10:15.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: local dir=td/osd-backfill-prio 2020-09-12T14:10:15.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: shift 2020-09-12T14:10:15.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: local id=x 2020-09-12T14:10:15.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: shift 2020-09-12T14:10:15.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: local data=td/osd-backfill-prio/x 2020-09-12T14:10:15.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:558: run_mgr: ceph config set mgr mgr/devicehealth/enable_monitoring off --force 2020-09-12T14:10:16.287 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: get_asok_path 2020-09-12T14:10:16.287 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:10:16.287 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:10:16.287 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:10:16.288 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:10:16.288 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:10:16.288 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.73258/$cluster-$name.asok' 2020-09-12T14:10:16.288 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:10:16.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-backfill-prio/x '--log-file=td/osd-backfill-prio/$name.log' '--admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' --run-dir=td/osd-backfill-prio '--pid-file=td/osd-backfill-prio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:10:16.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:332: TEST_backfill_pool_priority: export CEPH_ARGS 2020-09-12T14:10:16.312 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:334: TEST_backfill_pool_priority: expr 2 - 1 2020-09-12T14:10:16.313 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:334: TEST_backfill_pool_priority: seq 0 1 2020-09-12T14:10:16.316 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:334: TEST_backfill_pool_priority: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:10:16.316 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:336: TEST_backfill_pool_priority: run_osd td/osd-backfill-prio 0 2020-09-12T14:10:16.316 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-prio 2020-09-12T14:10:16.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:10:16.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=0 2020-09-12T14:10:16.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:10:16.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-prio/0 2020-09-12T14:10:16.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=576afb9f-26d0-41c8-a49c-302fc5ab0911 --auth-supported=none --mon-host=127.0.0.1:7114 --osd_max_backfills=1 --debug_reserver=20 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 ' 2020-09-12T14:10:16.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:10:16.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:10:16.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:10:16.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-prio/0' 2020-09-12T14:10:16.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-prio/0/journal' 2020-09-12T14:10:16.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:10:16.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:10:16.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-prio' 2020-09-12T14:10:16.318 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:10:16.319 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:10:16.319 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:10:16.319 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:10:16.319 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:10:16.319 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:10:16.319 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.73258/$cluster-$name.asok' 2020-09-12T14:10:16.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' 2020-09-12T14:10:16.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:10:16.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:10:16.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:10:16.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-prio/$name.log' 2020-09-12T14:10:16.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-prio/$name.pid' 2020-09-12T14:10:16.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:10:16.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:10:16.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:10:16.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:10:16.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:10:16.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-prio/0 2020-09-12T14:10:16.321 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:10:16.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=f3ec905c-7f96-4407-a60e-59f74066723e 2020-09-12T14:10:16.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd0 f3ec905c-7f96-4407-a60e-59f74066723e' 2020-09-12T14:10:16.322 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:10:16.322 INFO:tasks.workunit.client.0.smithi027.stdout:add osd0 f3ec905c-7f96-4407-a60e-59f74066723e 2020-09-12T14:10:16.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDI1lxfridtFBAA+ZrivifyRMKe/LiW5ciUcw== 2020-09-12T14:10:16.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDI1lxfridtFBAA+ZrivifyRMKe/LiW5ciUcw=="}' 2020-09-12T14:10:16.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new f3ec905c-7f96-4407-a60e-59f74066723e -i td/osd-backfill-prio/0/new.json 2020-09-12T14:10:16.718 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:10:16.728 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-prio/0/new.json 2020-09-12T14:10:16.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 0 --fsid=576afb9f-26d0-41c8-a49c-302fc5ab0911 --auth-supported=none --mon-host=127.0.0.1:7114 --osd_max_backfills=1 --debug_reserver=20 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-prio/0 --osd-journal=td/osd-backfill-prio/0/journal --chdir= --run-dir=td/osd-backfill-prio '--admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-prio/$name.log' '--pid-file=td/osd-backfill-prio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDI1lxfridtFBAA+ZrivifyRMKe/LiW5ciUcw== --osd-uuid f3ec905c-7f96-4407-a60e-59f74066723e 2020-09-12T14:10:16.757 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:16.755+0000 7f4b85ba6ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:16.759 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:16.757+0000 7f4b85ba6ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:16.761 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:16.759+0000 7f4b85ba6ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:16.762 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:16.760+0000 7f4b85ba6ec0 -1 bluestore(td/osd-backfill-prio/0/block) _read_bdev_label failed to open td/osd-backfill-prio/0/block: (2) No such file or directory 2020-09-12T14:10:16.762 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:16.760+0000 7f4b85ba6ec0 -1 bluestore(td/osd-backfill-prio/0/block) _read_bdev_label failed to open td/osd-backfill-prio/0/block: (2) No such file or directory 2020-09-12T14:10:16.762 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:16.760+0000 7f4b85ba6ec0 -1 bluestore(td/osd-backfill-prio/0/block) _read_bdev_label failed to open td/osd-backfill-prio/0/block: (2) No such file or directory 2020-09-12T14:10:16.815 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:16.813+0000 7f4b85ba6ec0 -1 bluestore(td/osd-backfill-prio/0) _read_fsid unparsable uuid 2020-09-12T14:10:21.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-prio/0/keyring 2020-09-12T14:10:21.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:10:21.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd0 key to auth repository 2020-09-12T14:10:21.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-prio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:10:21.034 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd0 key to auth repository 2020-09-12T14:10:21.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.0 2020-09-12T14:10:21.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 0 --fsid=576afb9f-26d0-41c8-a49c-302fc5ab0911 --auth-supported=none --mon-host=127.0.0.1:7114 --osd_max_backfills=1 --debug_reserver=20 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-prio/0 --osd-journal=td/osd-backfill-prio/0/journal --chdir= --run-dir=td/osd-backfill-prio '--admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-prio/$name.log' '--pid-file=td/osd-backfill-prio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:10:21.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:10:21.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:10:21.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:10:21.580 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.0 2020-09-12T14:10:21.605 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:21.603+0000 7f35daf7aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:21.630 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:21.629+0000 7f35daf7aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:21.632 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:21.630+0000 7f35daf7aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:21.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 0 2020-09-12T14:10:21.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:10:21.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T14:10:21.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:10:21.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:10:21.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:10:21.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:10:21.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:10:21.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:10:21.989 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:10:22.147 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:22.146+0000 7f35daf7aec0 -1 Falling back to public interface 2020-09-12T14:10:22.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:10:22.956 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:22.955+0000 7f35daf7aec0 -1 osd.0 0 log_to_monitors {default=true} 2020-09-12T14:10:23.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:10:23.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:10:23.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:10:23.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:10:23.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:10:23.411 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:10:23.832 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/74119,v1:127.0.0.1:6803/74119] [v2:127.0.0.1:6804/74119,v1:127.0.0.1:6805/74119] exists,up f3ec905c-7f96-4407-a60e-59f74066723e 2020-09-12T14:10:23.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:10:23.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:10:23.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:10:23.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:334: TEST_backfill_pool_priority: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:10:23.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:336: TEST_backfill_pool_priority: run_osd td/osd-backfill-prio 1 2020-09-12T14:10:23.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-prio 2020-09-12T14:10:23.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:10:23.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=1 2020-09-12T14:10:23.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:10:23.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-prio/1 2020-09-12T14:10:23.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=576afb9f-26d0-41c8-a49c-302fc5ab0911 --auth-supported=none --mon-host=127.0.0.1:7114 --osd_max_backfills=1 --debug_reserver=20 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 ' 2020-09-12T14:10:23.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:10:23.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:10:23.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:10:23.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-prio/1' 2020-09-12T14:10:23.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-prio/1/journal' 2020-09-12T14:10:23.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:10:23.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:10:23.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-prio' 2020-09-12T14:10:23.835 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:10:23.835 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:10:23.835 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:10:23.835 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:10:23.836 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:10:23.836 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:10:23.836 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.73258/$cluster-$name.asok' 2020-09-12T14:10:23.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' 2020-09-12T14:10:23.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:10:23.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:10:23.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:10:23.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-prio/$name.log' 2020-09-12T14:10:23.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-prio/$name.pid' 2020-09-12T14:10:23.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:10:23.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:10:23.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:10:23.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:10:23.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:10:23.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-prio/1 2020-09-12T14:10:23.838 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:10:23.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=c9e25c48-85d7-4ce3-88c8-0cde44bbf688 2020-09-12T14:10:23.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd1 c9e25c48-85d7-4ce3-88c8-0cde44bbf688' 2020-09-12T14:10:23.839 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:10:23.839 INFO:tasks.workunit.client.0.smithi027.stdout:add osd1 c9e25c48-85d7-4ce3-88c8-0cde44bbf688 2020-09-12T14:10:23.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDP1lxfmPmiMhAAdaZFHPavzLK2O4ERi8mxCw== 2020-09-12T14:10:23.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDP1lxfmPmiMhAAdaZFHPavzLK2O4ERi8mxCw=="}' 2020-09-12T14:10:23.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new c9e25c48-85d7-4ce3-88c8-0cde44bbf688 -i td/osd-backfill-prio/1/new.json 2020-09-12T14:10:24.362 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:10:24.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-prio/1/new.json 2020-09-12T14:10:24.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 1 --fsid=576afb9f-26d0-41c8-a49c-302fc5ab0911 --auth-supported=none --mon-host=127.0.0.1:7114 --osd_max_backfills=1 --debug_reserver=20 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-prio/1 --osd-journal=td/osd-backfill-prio/1/journal --chdir= --run-dir=td/osd-backfill-prio '--admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-prio/$name.log' '--pid-file=td/osd-backfill-prio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDP1lxfmPmiMhAAdaZFHPavzLK2O4ERi8mxCw== --osd-uuid c9e25c48-85d7-4ce3-88c8-0cde44bbf688 2020-09-12T14:10:24.403 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:24.401+0000 7f76cdc10ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:24.430 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:24.429+0000 7f76cdc10ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:24.431 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:24.429+0000 7f76cdc10ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:24.432 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:24.430+0000 7f76cdc10ec0 -1 bluestore(td/osd-backfill-prio/1/block) _read_bdev_label failed to open td/osd-backfill-prio/1/block: (2) No such file or directory 2020-09-12T14:10:24.432 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:24.430+0000 7f76cdc10ec0 -1 bluestore(td/osd-backfill-prio/1/block) _read_bdev_label failed to open td/osd-backfill-prio/1/block: (2) No such file or directory 2020-09-12T14:10:24.432 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:24.430+0000 7f76cdc10ec0 -1 bluestore(td/osd-backfill-prio/1/block) _read_bdev_label failed to open td/osd-backfill-prio/1/block: (2) No such file or directory 2020-09-12T14:10:24.488 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:24.487+0000 7f76cdc10ec0 -1 bluestore(td/osd-backfill-prio/1) _read_fsid unparsable uuid 2020-09-12T14:10:28.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-prio/1/keyring 2020-09-12T14:10:28.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:10:28.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd1 key to auth repository 2020-09-12T14:10:28.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-prio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:10:28.696 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd1 key to auth repository 2020-09-12T14:10:29.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.1 2020-09-12T14:10:29.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 1 --fsid=576afb9f-26d0-41c8-a49c-302fc5ab0911 --auth-supported=none --mon-host=127.0.0.1:7114 --osd_max_backfills=1 --debug_reserver=20 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-prio/1 --osd-journal=td/osd-backfill-prio/1/journal --chdir= --run-dir=td/osd-backfill-prio '--admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-prio/$name.log' '--pid-file=td/osd-backfill-prio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:10:29.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:10:29.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:10:29.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:10:29.249 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T14:10:29.273 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:29.273+0000 7f48f16a2ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:29.294 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:29.293+0000 7f48f16a2ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:29.295 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:29.294+0000 7f48f16a2ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:10:29.653 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 1 2020-09-12T14:10:29.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:10:29.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:10:29.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:10:29.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:10:29.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:10:29.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:10:29.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:10:29.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:10:29.655 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:10:29.810 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:29.810+0000 7f48f16a2ec0 -1 Falling back to public interface 2020-09-12T14:10:30.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:10:30.676 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:10:30.676+0000 7f48f16a2ec0 -1 osd.1 0 log_to_monitors {default=true} 2020-09-12T14:10:31.089 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:10:31.089 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:10:31.089 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:10:31.090 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:10:31.090 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:10:31.090 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:10:31.509 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/75113,v1:127.0.0.1:6811/75113] [v2:127.0.0.1:6812/75113,v1:127.0.0.1:6813/75113] exists,up c9e25c48-85d7-4ce3-88c8-0cde44bbf688 2020-09-12T14:10:31.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:10:31.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:10:31.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:10:31.510 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:339: TEST_backfill_pool_priority: seq 1 3 2020-09-12T14:10:31.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:339: TEST_backfill_pool_priority: for p in $(seq 1 $pools) 2020-09-12T14:10:31.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:341: TEST_backfill_pool_priority: create_pool test1 1 1 2020-09-12T14:10:31.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test1 1 1 2020-09-12T14:10:32.170 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test1' already exists 2020-09-12T14:10:32.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:10:33.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:342: TEST_backfill_pool_priority: ceph osd pool set test1 size 2 2020-09-12T14:10:33.921 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 1 size to 2 2020-09-12T14:10:33.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:339: TEST_backfill_pool_priority: for p in $(seq 1 $pools) 2020-09-12T14:10:33.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:341: TEST_backfill_pool_priority: create_pool test2 1 1 2020-09-12T14:10:33.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test2 1 1 2020-09-12T14:10:34.588 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test2' already exists 2020-09-12T14:10:34.597 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:10:35.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:342: TEST_backfill_pool_priority: ceph osd pool set test2 size 2 2020-09-12T14:10:36.557 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 2 size to 2 2020-09-12T14:10:36.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:339: TEST_backfill_pool_priority: for p in $(seq 1 $pools) 2020-09-12T14:10:36.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:341: TEST_backfill_pool_priority: create_pool test3 1 1 2020-09-12T14:10:36.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test3 1 1 2020-09-12T14:10:37.275 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test3' already exists 2020-09-12T14:10:37.284 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:10:38.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:342: TEST_backfill_pool_priority: ceph osd pool set test3 size 2 2020-09-12T14:10:39.175 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 3 size to 2 2020-09-12T14:10:39.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:344: TEST_backfill_pool_priority: sleep 5 2020-09-12T14:10:44.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:346: TEST_backfill_pool_priority: wait_for_clean 2020-09-12T14:10:44.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:10:44.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:10:44.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:10:44.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:10:44.187 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 90 .1 2020-09-12T14:10:44.187 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:10:44.187 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:10:44.187 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:10:44.188 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:10:44.188 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:10:44.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:10:44.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:10:44.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:10:44.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:10:44.293 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:10:44.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:10:44.704 INFO:tasks.workunit.client.0.smithi027.stderr:1' 2020-09-12T14:10:44.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:10:44.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:10:44.704 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:10:44.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836488 2020-09-12T14:10:44.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836488 2020-09-12T14:10:44.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836488' 2020-09-12T14:10:44.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:10:44.851 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:10:44.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672965 2020-09-12T14:10:44.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672965 2020-09-12T14:10:44.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836488 1-42949672965' 2020-09-12T14:10:44.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:10:44.996 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836488 2020-09-12T14:10:44.996 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:10:44.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:10:44.997 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836488 2020-09-12T14:10:44.998 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:10:44.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836488 2020-09-12T14:10:44.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836488' 2020-09-12T14:10:44.999 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836488 2020-09-12T14:10:44.999 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:10:45.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836485 -lt 21474836488 2020-09-12T14:10:45.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:10:46.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:10:46.422 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:10:46.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836488 -lt 21474836488 2020-09-12T14:10:46.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:10:46.835 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672965 2020-09-12T14:10:46.835 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:10:46.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:10:46.837 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672965 2020-09-12T14:10:46.837 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:10:46.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672965 2020-09-12T14:10:46.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672965' 2020-09-12T14:10:46.838 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672965 2020-09-12T14:10:46.839 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:10:47.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672965 -lt 42949672965 2020-09-12T14:10:47.288 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:10:47.289 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:10:47.289 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:10:47.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 3 == 0 2020-09-12T14:10:47.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:10:47.828 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:10:47.828 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:10:47.828 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:10:47.828 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:10:47.829 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:10:47.829 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:10:48.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=3 2020-09-12T14:10:48.224 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:10:48.225 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:10:48.225 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:10:48.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 3 = 3 2020-09-12T14:10:48.765 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T14:10:48.765 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T14:10:48.765 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:348: TEST_backfill_pool_priority: ceph pg dump pgs 2020-09-12T14:10:49.149 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:10:49.149 INFO:tasks.workunit.client.0.smithi027.stdout:3.0 0 0 0 0 0 0 0 0 0 0 active+clean 2020-09-12T14:10:39.598187+0000 0'0 27:17 [1,0] 1 [1,0] 1 0'0 2020-09-12T14:10:37.021709+0000 0'0 2020-09-12T14:10:37.021709+0000 0 2020-09-12T14:10:49.149 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 0 0 0 0 0 0 0 0 0 0 active+clean 2020-09-12T14:10:34.236005+0000 0'0 26:27 [1,0] 1 [1,0] 1 0'0 2020-09-12T14:10:31.951228+0000 0'0 2020-09-12T14:10:31.951228+0000 0 2020-09-12T14:10:49.149 INFO:tasks.workunit.client.0.smithi027.stdout:2.0 0 0 0 0 0 0 0 0 0 0 active+clean 2020-09-12T14:10:36.863177+0000 0'0 26:22 [0,1] 0 [0,1] 0 0'0 2020-09-12T14:10:34.278646+0000 0'0 2020-09-12T14:10:34.278646+0000 0 2020-09-12T14:10:49.149 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:10:49.150 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:10:49.150 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:10:49.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:352: TEST_backfill_pool_priority: local PG1 2020-09-12T14:10:49.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:353: TEST_backfill_pool_priority: local POOLNUM1 2020-09-12T14:10:49.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:354: TEST_backfill_pool_priority: local pool1 2020-09-12T14:10:49.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:355: TEST_backfill_pool_priority: local chk_osd1_1 2020-09-12T14:10:49.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:356: TEST_backfill_pool_priority: local chk_osd1_2 2020-09-12T14:10:49.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:358: TEST_backfill_pool_priority: local PG2 2020-09-12T14:10:49.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:359: TEST_backfill_pool_priority: local POOLNUM2 2020-09-12T14:10:49.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:360: TEST_backfill_pool_priority: local pool2 2020-09-12T14:10:49.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:361: TEST_backfill_pool_priority: local chk_osd2_1 2020-09-12T14:10:49.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:362: TEST_backfill_pool_priority: local chk_osd2_2 2020-09-12T14:10:49.162 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:364: TEST_backfill_pool_priority: seq 1 3 2020-09-12T14:10:49.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:364: TEST_backfill_pool_priority: for p in $(seq 1 $pools) 2020-09-12T14:10:49.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:366: TEST_backfill_pool_priority: ceph pg map 1.0 --format=json 2020-09-12T14:10:49.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:366: TEST_backfill_pool_priority: jq '.acting[]' 2020-09-12T14:10:49.599 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:367: TEST_backfill_pool_priority: head -1 td/osd-backfill-prio/acting 2020-09-12T14:10:49.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:367: TEST_backfill_pool_priority: local test_osd1=1 2020-09-12T14:10:49.601 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:368: TEST_backfill_pool_priority: tail -1 td/osd-backfill-prio/acting 2020-09-12T14:10:49.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:368: TEST_backfill_pool_priority: local test_osd2=0 2020-09-12T14:10:49.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:369: TEST_backfill_pool_priority: '[' -z '' ']' 2020-09-12T14:10:49.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:371: TEST_backfill_pool_priority: PG1=1.0 2020-09-12T14:10:49.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:372: TEST_backfill_pool_priority: POOLNUM1=1 2020-09-12T14:10:49.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:373: TEST_backfill_pool_priority: pool1=test1 2020-09-12T14:10:49.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:374: TEST_backfill_pool_priority: chk_osd1_1=1 2020-09-12T14:10:49.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:375: TEST_backfill_pool_priority: chk_osd1_2=0 2020-09-12T14:10:49.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:364: TEST_backfill_pool_priority: for p in $(seq 1 $pools) 2020-09-12T14:10:49.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:366: TEST_backfill_pool_priority: ceph pg map 2.0 --format=json 2020-09-12T14:10:49.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:366: TEST_backfill_pool_priority: jq '.acting[]' 2020-09-12T14:10:50.001 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:367: TEST_backfill_pool_priority: head -1 td/osd-backfill-prio/acting 2020-09-12T14:10:50.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:367: TEST_backfill_pool_priority: local test_osd1=0 2020-09-12T14:10:50.003 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:368: TEST_backfill_pool_priority: tail -1 td/osd-backfill-prio/acting 2020-09-12T14:10:50.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:368: TEST_backfill_pool_priority: local test_osd2=1 2020-09-12T14:10:50.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:369: TEST_backfill_pool_priority: '[' -z 1.0 ']' 2020-09-12T14:10:50.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:376: TEST_backfill_pool_priority: '[' 1 '!=' 0 ']' 2020-09-12T14:10:50.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:378: TEST_backfill_pool_priority: PG2=2.0 2020-09-12T14:10:50.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:379: TEST_backfill_pool_priority: POOLNUM2=2 2020-09-12T14:10:50.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:380: TEST_backfill_pool_priority: pool2=test2 2020-09-12T14:10:50.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:381: TEST_backfill_pool_priority: chk_osd2_1=0 2020-09-12T14:10:50.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:382: TEST_backfill_pool_priority: chk_osd2_2=1 2020-09-12T14:10:50.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:383: TEST_backfill_pool_priority: break 2020-09-12T14:10:50.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:386: TEST_backfill_pool_priority: rm -f td/osd-backfill-prio/acting 2020-09-12T14:10:50.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:388: TEST_backfill_pool_priority: '[' test2 = '' ']' 2020-09-12T14:10:50.006 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:394: TEST_backfill_pool_priority: seq 1 3 2020-09-12T14:10:50.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:394: TEST_backfill_pool_priority: for p in $(seq 1 $pools) 2020-09-12T14:10:50.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:396: TEST_backfill_pool_priority: '[' 1 '!=' 1 -a 1 '!=' 2 ']' 2020-09-12T14:10:50.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:394: TEST_backfill_pool_priority: for p in $(seq 1 $pools) 2020-09-12T14:10:50.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:396: TEST_backfill_pool_priority: '[' 2 '!=' 1 -a 2 '!=' 2 ']' 2020-09-12T14:10:50.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:394: TEST_backfill_pool_priority: for p in $(seq 1 $pools) 2020-09-12T14:10:50.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:396: TEST_backfill_pool_priority: '[' 3 '!=' 1 -a 3 '!=' 2 ']' 2020-09-12T14:10:50.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:398: TEST_backfill_pool_priority: delete_pool test3 2020-09-12T14:10:50.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: delete_pool: local poolname=test3 2020-09-12T14:10:50.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: ceph osd pool delete test3 test3 --yes-i-really-really-mean-it 2020-09-12T14:10:50.539 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test3' does not exist 2020-09-12T14:10:50.548 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:402: TEST_backfill_pool_priority: pool1_extra_prio=1 2020-09-12T14:10:50.548 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:403: TEST_backfill_pool_priority: pool2_extra_prio=2 2020-09-12T14:10:50.549 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:405: TEST_backfill_pool_priority: expr 150 + 1 + 1 2020-09-12T14:10:50.550 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:405: TEST_backfill_pool_priority: pool1_prio=152 2020-09-12T14:10:50.551 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:406: TEST_backfill_pool_priority: expr 150 + 1 + 2 2020-09-12T14:10:50.551 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:406: TEST_backfill_pool_priority: pool2_prio=153 2020-09-12T14:10:50.552 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:408: TEST_backfill_pool_priority: ceph osd pool set test1 size 1 --yes-i-really-mean-it 2020-09-12T14:10:51.261 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 1 size to 1 2020-09-12T14:10:51.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:409: TEST_backfill_pool_priority: ceph osd pool set test1 recovery_priority 1 2020-09-12T14:10:52.031 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 1 recovery_priority to 1 2020-09-12T14:10:52.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:410: TEST_backfill_pool_priority: ceph osd pool set test2 size 1 --yes-i-really-mean-it 2020-09-12T14:10:52.726 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 2 size to 1 2020-09-12T14:10:52.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:411: TEST_backfill_pool_priority: ceph osd pool set test2 recovery_priority 2 2020-09-12T14:10:53.412 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 2 recovery_priority to 2 2020-09-12T14:10:53.422 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:412: TEST_backfill_pool_priority: wait_for_clean 2020-09-12T14:10:53.422 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:10:53.422 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:10:53.422 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:10:53.422 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:10:53.423 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 90 .1 2020-09-12T14:10:53.423 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:10:53.423 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:10:53.424 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:10:53.424 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:10:53.424 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:10:53.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:10:53.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:10:53.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:10:53.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:10:53.516 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:10:53.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:10:53.970 INFO:tasks.workunit.client.0.smithi027.stderr:1' 2020-09-12T14:10:53.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:10:53.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:10:53.970 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:10:54.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836491 2020-09-12T14:10:54.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836491 2020-09-12T14:10:54.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836491' 2020-09-12T14:10:54.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:10:54.129 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:10:54.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672969 2020-09-12T14:10:54.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672969 2020-09-12T14:10:54.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836491 1-42949672969' 2020-09-12T14:10:54.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:10:54.274 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836491 2020-09-12T14:10:54.274 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:10:54.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:10:54.276 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836491 2020-09-12T14:10:54.276 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:10:54.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836491 2020-09-12T14:10:54.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2020-09-12T14:10:54.277 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836491 2020-09-12T14:10:54.277 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:10:54.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836489 -lt 21474836491 2020-09-12T14:10:54.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:10:55.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:10:55.687 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:10:56.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836489 -lt 21474836491 2020-09-12T14:10:56.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:10:57.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 299 -eq 0 ']' 2020-09-12T14:10:57.103 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:10:57.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836492 -lt 21474836491 2020-09-12T14:10:57.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:10:57.524 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672969 2020-09-12T14:10:57.524 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:10:57.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:10:57.526 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672969 2020-09-12T14:10:57.526 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:10:57.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672969 2020-09-12T14:10:57.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672969' 2020-09-12T14:10:57.527 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672969 2020-09-12T14:10:57.528 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:10:57.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672969 -lt 42949672969 2020-09-12T14:10:57.957 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:10:57.958 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:10:57.958 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:10:58.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 2 == 0 2020-09-12T14:10:58.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:10:58.502 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:10:58.502 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:10:58.502 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:10:58.502 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:10:58.502 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:10:58.503 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:10:59.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=2 2020-09-12T14:10:59.006 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:10:59.007 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:10:59.007 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:10:59.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 2 = 2 2020-09-12T14:10:59.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T14:10:59.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T14:10:59.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:414: TEST_backfill_pool_priority: dd if=/dev/urandom of=td/osd-backfill-prio/data bs=1M count=10 2020-09-12T14:10:59.592 INFO:tasks.workunit.client.0.smithi027.stderr:10+0 records in 2020-09-12T14:10:59.592 INFO:tasks.workunit.client.0.smithi027.stderr:10+0 records out 2020-09-12T14:10:59.593 INFO:tasks.workunit.client.0.smithi027.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0464717 s, 226 MB/s 2020-09-12T14:10:59.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:415: TEST_backfill_pool_priority: p=1 2020-09-12T14:10:59.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:416: TEST_backfill_pool_priority: for pname in $pool1 $pool2 2020-09-12T14:10:59.593 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: seq 1 50 2020-09-12T14:10:59.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:10:59.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj1-p1 td/osd-backfill-prio/data 2020-09-12T14:10:59.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:10:59.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj2-p1 td/osd-backfill-prio/data 2020-09-12T14:11:00.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:00.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj3-p1 td/osd-backfill-prio/data 2020-09-12T14:11:00.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:00.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj4-p1 td/osd-backfill-prio/data 2020-09-12T14:11:01.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:01.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj5-p1 td/osd-backfill-prio/data 2020-09-12T14:11:01.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:01.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj6-p1 td/osd-backfill-prio/data 2020-09-12T14:11:01.932 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:01.932 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj7-p1 td/osd-backfill-prio/data 2020-09-12T14:11:02.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:02.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj8-p1 td/osd-backfill-prio/data 2020-09-12T14:11:02.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:02.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj9-p1 td/osd-backfill-prio/data 2020-09-12T14:11:03.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:03.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj10-p1 td/osd-backfill-prio/data 2020-09-12T14:11:03.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:03.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj11-p1 td/osd-backfill-prio/data 2020-09-12T14:11:03.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:03.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj12-p1 td/osd-backfill-prio/data 2020-09-12T14:11:04.663 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:04.664 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj13-p1 td/osd-backfill-prio/data 2020-09-12T14:11:04.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:04.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj14-p1 td/osd-backfill-prio/data 2020-09-12T14:11:05.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:05.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj15-p1 td/osd-backfill-prio/data 2020-09-12T14:11:05.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:05.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj16-p1 td/osd-backfill-prio/data 2020-09-12T14:11:05.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:05.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj17-p1 td/osd-backfill-prio/data 2020-09-12T14:11:06.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:06.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj18-p1 td/osd-backfill-prio/data 2020-09-12T14:11:06.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:06.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj19-p1 td/osd-backfill-prio/data 2020-09-12T14:11:07.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:07.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj20-p1 td/osd-backfill-prio/data 2020-09-12T14:11:07.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:07.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj21-p1 td/osd-backfill-prio/data 2020-09-12T14:11:08.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:08.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj22-p1 td/osd-backfill-prio/data 2020-09-12T14:11:08.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:08.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj23-p1 td/osd-backfill-prio/data 2020-09-12T14:11:08.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:08.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj24-p1 td/osd-backfill-prio/data 2020-09-12T14:11:09.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:09.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj25-p1 td/osd-backfill-prio/data 2020-09-12T14:11:10.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:10.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj26-p1 td/osd-backfill-prio/data 2020-09-12T14:11:10.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:10.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj27-p1 td/osd-backfill-prio/data 2020-09-12T14:11:11.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:11.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj28-p1 td/osd-backfill-prio/data 2020-09-12T14:11:11.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:11.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj29-p1 td/osd-backfill-prio/data 2020-09-12T14:11:12.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:12.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj30-p1 td/osd-backfill-prio/data 2020-09-12T14:11:12.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:12.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj31-p1 td/osd-backfill-prio/data 2020-09-12T14:11:12.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:12.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj32-p1 td/osd-backfill-prio/data 2020-09-12T14:11:13.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:13.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj33-p1 td/osd-backfill-prio/data 2020-09-12T14:11:13.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:13.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj34-p1 td/osd-backfill-prio/data 2020-09-12T14:11:13.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:13.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj35-p1 td/osd-backfill-prio/data 2020-09-12T14:11:14.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:14.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj36-p1 td/osd-backfill-prio/data 2020-09-12T14:11:15.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:15.032 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj37-p1 td/osd-backfill-prio/data 2020-09-12T14:11:15.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:15.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj38-p1 td/osd-backfill-prio/data 2020-09-12T14:11:15.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:15.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj39-p1 td/osd-backfill-prio/data 2020-09-12T14:11:16.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:16.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj40-p1 td/osd-backfill-prio/data 2020-09-12T14:11:16.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:16.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj41-p1 td/osd-backfill-prio/data 2020-09-12T14:11:16.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:16.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj42-p1 td/osd-backfill-prio/data 2020-09-12T14:11:17.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:17.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj43-p1 td/osd-backfill-prio/data 2020-09-12T14:11:18.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:18.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj44-p1 td/osd-backfill-prio/data 2020-09-12T14:11:18.573 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:18.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj45-p1 td/osd-backfill-prio/data 2020-09-12T14:11:18.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:18.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj46-p1 td/osd-backfill-prio/data 2020-09-12T14:11:19.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:19.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj47-p1 td/osd-backfill-prio/data 2020-09-12T14:11:19.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:19.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj48-p1 td/osd-backfill-prio/data 2020-09-12T14:11:20.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:20.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj49-p1 td/osd-backfill-prio/data 2020-09-12T14:11:21.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:21.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test1 put obj50-p1 td/osd-backfill-prio/data 2020-09-12T14:11:21.588 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:422: TEST_backfill_pool_priority: expr 1 + 1 2020-09-12T14:11:21.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:422: TEST_backfill_pool_priority: p=2 2020-09-12T14:11:21.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:416: TEST_backfill_pool_priority: for pname in $pool1 $pool2 2020-09-12T14:11:21.590 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: seq 1 50 2020-09-12T14:11:21.590 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:21.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj1-p2 td/osd-backfill-prio/data 2020-09-12T14:11:21.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:21.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj2-p2 td/osd-backfill-prio/data 2020-09-12T14:11:22.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:22.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj3-p2 td/osd-backfill-prio/data 2020-09-12T14:11:22.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:22.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj4-p2 td/osd-backfill-prio/data 2020-09-12T14:11:22.934 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:22.934 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj5-p2 td/osd-backfill-prio/data 2020-09-12T14:11:23.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:23.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj6-p2 td/osd-backfill-prio/data 2020-09-12T14:11:24.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:24.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj7-p2 td/osd-backfill-prio/data 2020-09-12T14:11:24.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:24.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj8-p2 td/osd-backfill-prio/data 2020-09-12T14:11:24.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:24.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj9-p2 td/osd-backfill-prio/data 2020-09-12T14:11:25.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:25.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj10-p2 td/osd-backfill-prio/data 2020-09-12T14:11:25.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:25.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj11-p2 td/osd-backfill-prio/data 2020-09-12T14:11:25.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:25.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj12-p2 td/osd-backfill-prio/data 2020-09-12T14:11:26.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:26.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj13-p2 td/osd-backfill-prio/data 2020-09-12T14:11:27.035 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:27.035 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj14-p2 td/osd-backfill-prio/data 2020-09-12T14:11:27.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:27.353 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj15-p2 td/osd-backfill-prio/data 2020-09-12T14:11:27.678 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:27.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj16-p2 td/osd-backfill-prio/data 2020-09-12T14:11:27.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:27.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj17-p2 td/osd-backfill-prio/data 2020-09-12T14:11:28.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:28.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj18-p2 td/osd-backfill-prio/data 2020-09-12T14:11:28.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:28.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj19-p2 td/osd-backfill-prio/data 2020-09-12T14:11:29.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:29.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj20-p2 td/osd-backfill-prio/data 2020-09-12T14:11:29.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:29.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj21-p2 td/osd-backfill-prio/data 2020-09-12T14:11:30.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:30.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj22-p2 td/osd-backfill-prio/data 2020-09-12T14:11:30.644 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:30.644 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj23-p2 td/osd-backfill-prio/data 2020-09-12T14:11:31.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:31.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj24-p2 td/osd-backfill-prio/data 2020-09-12T14:11:31.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:31.914 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj25-p2 td/osd-backfill-prio/data 2020-09-12T14:11:32.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:32.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj26-p2 td/osd-backfill-prio/data 2020-09-12T14:11:33.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:33.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj27-p2 td/osd-backfill-prio/data 2020-09-12T14:11:33.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:33.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj28-p2 td/osd-backfill-prio/data 2020-09-12T14:11:33.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:33.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj29-p2 td/osd-backfill-prio/data 2020-09-12T14:11:34.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:34.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj30-p2 td/osd-backfill-prio/data 2020-09-12T14:11:34.378 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:34.378 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj31-p2 td/osd-backfill-prio/data 2020-09-12T14:11:34.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:34.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj32-p2 td/osd-backfill-prio/data 2020-09-12T14:11:35.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:35.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj33-p2 td/osd-backfill-prio/data 2020-09-12T14:11:35.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:35.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj34-p2 td/osd-backfill-prio/data 2020-09-12T14:11:36.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:36.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj35-p2 td/osd-backfill-prio/data 2020-09-12T14:11:36.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:36.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj36-p2 td/osd-backfill-prio/data 2020-09-12T14:11:37.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:37.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj37-p2 td/osd-backfill-prio/data 2020-09-12T14:11:37.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:37.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj38-p2 td/osd-backfill-prio/data 2020-09-12T14:11:37.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:37.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj39-p2 td/osd-backfill-prio/data 2020-09-12T14:11:37.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:37.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj40-p2 td/osd-backfill-prio/data 2020-09-12T14:11:38.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:38.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj41-p2 td/osd-backfill-prio/data 2020-09-12T14:11:39.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:39.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj42-p2 td/osd-backfill-prio/data 2020-09-12T14:11:39.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:39.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj43-p2 td/osd-backfill-prio/data 2020-09-12T14:11:39.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:39.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj44-p2 td/osd-backfill-prio/data 2020-09-12T14:11:40.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:40.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj45-p2 td/osd-backfill-prio/data 2020-09-12T14:11:40.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:40.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj46-p2 td/osd-backfill-prio/data 2020-09-12T14:11:40.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:40.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj47-p2 td/osd-backfill-prio/data 2020-09-12T14:11:41.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:41.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj48-p2 td/osd-backfill-prio/data 2020-09-12T14:11:42.864 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:42.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj49-p2 td/osd-backfill-prio/data 2020-09-12T14:11:43.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:418: TEST_backfill_pool_priority: for i in $(seq 1 $objects) 2020-09-12T14:11:43.191 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:420: TEST_backfill_pool_priority: rados -p test2 put obj50-p2 td/osd-backfill-prio/data 2020-09-12T14:11:43.501 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:422: TEST_backfill_pool_priority: expr 2 + 1 2020-09-12T14:11:43.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:422: TEST_backfill_pool_priority: p=3 2020-09-12T14:11:43.502 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:425: TEST_backfill_pool_priority: get_not_primary test1 obj1-p1 2020-09-12T14:11:43.503 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1167: get_not_primary: local poolname=test1 2020-09-12T14:11:43.503 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: get_not_primary: local objectname=obj1-p1 2020-09-12T14:11:43.503 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1170: get_not_primary: get_primary test1 obj1-p1 2020-09-12T14:11:43.504 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1134: get_primary: local poolname=test1 2020-09-12T14:11:43.504 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1135: get_primary: local objectname=obj1-p1 2020-09-12T14:11:43.504 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1137: get_primary: ceph --format json osd map test1 obj1-p1 2020-09-12T14:11:43.504 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1138: get_primary: jq .acting_primary 2020-09-12T14:11:43.916 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1170: get_not_primary: local primary=1 2020-09-12T14:11:43.917 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1171: get_not_primary: ceph --format json osd map test1 obj1-p1 2020-09-12T14:11:43.917 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1172: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2020-09-12T14:11:44.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:425: TEST_backfill_pool_priority: local otherosd=null 2020-09-12T14:11:44.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:427: TEST_backfill_pool_priority: ceph pg dump pgs 2020-09-12T14:11:44.735 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:11:44.736 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 50 0 0 0 0 524288000 0 0 50 50 active+clean 2020-09-12T14:10:51.442748+0000 36'150 36:192 [1] 1 [1] 1 0'0 2020-09-12T14:10:31.951228+0000 0'0 2020-09-12T14:10:31.951228+0000 0 2020-09-12T14:11:44.736 INFO:tasks.workunit.client.0.smithi027.stdout:2.0 41 0 0 0 0 427819008 0 0 22 22 active+clean 2020-09-12T14:10:52.915567+0000 36'122 36:157 [0] 0 [0] 0 0'0 2020-09-12T14:10:34.278646+0000 0'0 2020-09-12T14:10:34.278646+0000 0 2020-09-12T14:11:44.736 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:11:44.736 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:11:44.737 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:11:44.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:428: TEST_backfill_pool_priority: ERRORS=0 2020-09-12T14:11:44.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:430: TEST_backfill_pool_priority: ceph osd pool set test1 size 2 2020-09-12T14:11:45.437 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 1 size to 2 2020-09-12T14:11:45.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:431: TEST_backfill_pool_priority: ceph osd pool set test2 size 2 2020-09-12T14:11:46.231 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 2 size to 2 2020-09-12T14:11:46.240 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:432: TEST_backfill_pool_priority: sleep 5 2020-09-12T14:11:51.243 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:433: TEST_backfill_pool_priority: get_asok_path osd.1 2020-09-12T14:11:51.243 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2020-09-12T14:11:51.244 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2020-09-12T14:11:51.244 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:11:51.244 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:11:51.244 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:11:51.244 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.73258/ceph-osd.1.asok 2020-09-12T14:11:51.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:433: TEST_backfill_pool_priority: CEPH_ARGS= 2020-09-12T14:11:51.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:433: TEST_backfill_pool_priority: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:11:51.364 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 2020-09-12T14:11:51.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:434: TEST_backfill_pool_priority: echo osd.1 2020-09-12T14:11:51.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:435: TEST_backfill_pool_priority: cat td/osd-backfill-prio/dump.1.out 2020-09-12T14:11:51.365 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:11:51.365 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:11:51.365 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:11:51.365 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:11:51.365 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:11:51.365 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:11:51.365 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:11:51.366 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:11:51.366 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 152, 2020-09-12T14:11:51.366 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:11:51.366 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:11:51.366 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:11:51.366 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:11:51.366 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:11:51.367 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:11:51.367 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:11:51.367 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:11:51.367 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:11:51.367 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:11:51.367 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "2.0", 2020-09-12T14:11:51.367 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 153, 2020-09-12T14:11:51.367 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:11:51.368 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:11:51.368 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:11:51.368 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:11:51.368 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:11:51.369 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:436: TEST_backfill_pool_priority: get_asok_path osd.0 2020-09-12T14:11:51.369 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2020-09-12T14:11:51.369 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2020-09-12T14:11:51.369 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:11:51.369 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:11:51.370 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:11:51.370 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.73258/ceph-osd.0.asok 2020-09-12T14:11:51.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:436: TEST_backfill_pool_priority: CEPH_ARGS= 2020-09-12T14:11:51.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:436: TEST_backfill_pool_priority: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.0.asok dump_recovery_reservations 2020-09-12T14:11:51.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:437: TEST_backfill_pool_priority: echo osd.0 2020-09-12T14:11:51.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:438: TEST_backfill_pool_priority: cat td/osd-backfill-prio/dump.0.out 2020-09-12T14:11:51.479 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 2020-09-12T14:11:51.479 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:11:51.479 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:11:51.480 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:11:51.480 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:11:51.480 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:11:51.480 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:11:51.480 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:11:51.480 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "2.0", 2020-09-12T14:11:51.480 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 153, 2020-09-12T14:11:51.481 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:11:51.481 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:11:51.481 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:11:51.481 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:11:51.481 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:11:51.481 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:11:51.481 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:11:51.481 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:11:51.482 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:11:51.482 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:11:51.482 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:11:51.482 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 152, 2020-09-12T14:11:51.482 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:11:51.482 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:11:51.482 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:11:51.483 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:11:51.483 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:11:51.483 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:441: TEST_backfill_pool_priority: cat td/osd-backfill-prio/dump.1.out 2020-09-12T14:11:51.484 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:441: TEST_backfill_pool_priority: jq '.local_reservations.in_progress[0].item' 2020-09-12T14:11:51.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:441: TEST_backfill_pool_priority: eval 'ITEM="1.0"' 2020-09-12T14:11:51.484 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:441: TEST_backfill_pool_priority: ITEM=1.0 2020-09-12T14:11:51.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:442: TEST_backfill_pool_priority: '[' 1.0 '!=' 1.0 ']' 2020-09-12T14:11:51.484 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:447: TEST_backfill_pool_priority: cat td/osd-backfill-prio/dump.1.out 2020-09-12T14:11:51.484 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:447: TEST_backfill_pool_priority: jq '.local_reservations.in_progress[0].prio' 2020-09-12T14:11:51.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:447: TEST_backfill_pool_priority: PRIO=152 2020-09-12T14:11:51.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:448: TEST_backfill_pool_priority: '[' 152 '!=' 152 ']' 2020-09-12T14:11:51.487 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:456: TEST_backfill_pool_priority: cat td/osd-backfill-prio/dump.0.out 2020-09-12T14:11:51.487 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:456: TEST_backfill_pool_priority: jq '.remote_reservations.in_progress[0].item' 2020-09-12T14:11:51.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:456: TEST_backfill_pool_priority: eval 'ITEM="1.0"' 2020-09-12T14:11:51.489 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:456: TEST_backfill_pool_priority: ITEM=1.0 2020-09-12T14:11:51.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:457: TEST_backfill_pool_priority: '[' 1.0 '!=' 1.0 ']' 2020-09-12T14:11:51.490 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:462: TEST_backfill_pool_priority: cat td/osd-backfill-prio/dump.0.out 2020-09-12T14:11:51.490 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:462: TEST_backfill_pool_priority: jq '.remote_reservations.in_progress[0].prio' 2020-09-12T14:11:51.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:462: TEST_backfill_pool_priority: PRIO=152 2020-09-12T14:11:51.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:463: TEST_backfill_pool_priority: '[' 152 '!=' 152 ']' 2020-09-12T14:11:51.493 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:471: TEST_backfill_pool_priority: cat td/osd-backfill-prio/dump.0.out 2020-09-12T14:11:51.494 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:471: TEST_backfill_pool_priority: jq '.local_reservations.in_progress[0].item' 2020-09-12T14:11:51.496 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:471: TEST_backfill_pool_priority: eval 'ITEM="2.0"' 2020-09-12T14:11:51.496 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:471: TEST_backfill_pool_priority: ITEM=2.0 2020-09-12T14:11:51.496 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:472: TEST_backfill_pool_priority: '[' 2.0 '!=' 2.0 ']' 2020-09-12T14:11:51.497 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:477: TEST_backfill_pool_priority: cat td/osd-backfill-prio/dump.0.out 2020-09-12T14:11:51.497 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:477: TEST_backfill_pool_priority: jq '.local_reservations.in_progress[0].prio' 2020-09-12T14:11:51.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:477: TEST_backfill_pool_priority: PRIO=153 2020-09-12T14:11:51.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:478: TEST_backfill_pool_priority: '[' 153 '!=' 153 ']' 2020-09-12T14:11:51.500 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:486: TEST_backfill_pool_priority: cat td/osd-backfill-prio/dump.1.out 2020-09-12T14:11:51.500 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:486: TEST_backfill_pool_priority: jq '.remote_reservations.in_progress[0].item' 2020-09-12T14:11:51.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:486: TEST_backfill_pool_priority: eval 'ITEM="2.0"' 2020-09-12T14:11:51.503 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:486: TEST_backfill_pool_priority: ITEM=2.0 2020-09-12T14:11:51.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:487: TEST_backfill_pool_priority: '[' 2.0 '!=' 2.0 ']' 2020-09-12T14:11:51.504 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:492: TEST_backfill_pool_priority: jq '.remote_reservations.in_progress[0].prio' 2020-09-12T14:11:51.504 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:492: TEST_backfill_pool_priority: cat td/osd-backfill-prio/dump.1.out 2020-09-12T14:11:51.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:492: TEST_backfill_pool_priority: PRIO=153 2020-09-12T14:11:51.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:493: TEST_backfill_pool_priority: '[' 153 '!=' 153 ']' 2020-09-12T14:11:51.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:500: TEST_backfill_pool_priority: wait_for_clean 2020-09-12T14:11:51.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:11:51.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:11:51.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:11:51.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:11:51.508 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 90 .1 2020-09-12T14:11:51.508 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:11:51.508 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:11:51.508 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:11:51.509 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:11:51.509 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:11:51.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:11:51.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:11:51.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:11:51.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:11:51.601 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:11:52.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:11:52.098 INFO:tasks.workunit.client.0.smithi027.stderr:1' 2020-09-12T14:11:52.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:11:52.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:11:52.099 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:11:52.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836505 2020-09-12T14:11:52.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836505 2020-09-12T14:11:52.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836505' 2020-09-12T14:11:52.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:11:52.307 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:11:52.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672983 2020-09-12T14:11:52.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672983 2020-09-12T14:11:52.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836505 1-42949672983' 2020-09-12T14:11:52.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:11:52.453 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836505 2020-09-12T14:11:52.454 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:11:52.454 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:11:52.455 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836505 2020-09-12T14:11:52.455 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:11:52.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836505 2020-09-12T14:11:52.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836505' 2020-09-12T14:11:52.457 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836505 2020-09-12T14:11:52.457 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:11:52.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836503 -lt 21474836505 2020-09-12T14:11:52.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:11:53.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:11:53.918 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:11:54.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836505 -lt 21474836505 2020-09-12T14:11:54.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:11:54.395 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672983 2020-09-12T14:11:54.395 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:11:54.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:11:54.397 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672983 2020-09-12T14:11:54.397 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:11:54.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672983 2020-09-12T14:11:54.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672983' 2020-09-12T14:11:54.398 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672983 2020-09-12T14:11:54.398 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:11:54.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672983 -lt 42949672983 2020-09-12T14:11:54.831 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:11:54.831 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:11:54.831 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:11:55.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 2 == 0 2020-09-12T14:11:55.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:11:55.393 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:11:55.393 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:11:55.393 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:11:55.394 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:11:55.394 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:11:55.394 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:11:55.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:11:55.804 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:11:55.805 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:11:55.805 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:11:56.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:11:56.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' -1 2020-09-12T14:11:56.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1598: wait_for_clean: loop=0 2020-09-12T14:11:56.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1599: wait_for_clean: num_active_clean=0 2020-09-12T14:11:56.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:11:56.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:11:56.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:11:56.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:11:56.711 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:11:56.712 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:11:56.712 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:11:56.712 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:11:56.712 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:11:56.712 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:11:57.110 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:11:57.111 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:11:57.111 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:11:57.112 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:11:57.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:11:57.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:11:57.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:11:57.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:11:57.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:11:57.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:11:57.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:11:57.801 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:11:57.802 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:11:58.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=15682490 2020-09-12T14:11:58.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 15682490 '!=' null 2020-09-12T14:11:58.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:11:58.431 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:11:58.431 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:11:58.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:11:58.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:11:58.532 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:11:58.532 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:11:58.532 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:11:58.532 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:11:58.532 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:11:58.533 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:11:58.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:11:58.959 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:11:58.959 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:11:58.959 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:11:59.510 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:11:59.510 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:11:59.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:11:59.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:11:59.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:11:59.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:11:59.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:11:59.512 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:11:59.512 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:00.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=15682490 2020-09-12T14:12:00.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 15682490 '!=' null 2020-09-12T14:12:00.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:00.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:00.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:00.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:00.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:00.270 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:00.270 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:00.270 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:00.270 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:00.270 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:00.270 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:00.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:00.735 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:00.736 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:00.736 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:01.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:01.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:01.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:01.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:01.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:01.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:01.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:01.278 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:01.278 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:01.887 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=20093508 2020-09-12T14:12:01.887 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 20093508 '!=' null 2020-09-12T14:12:01.887 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:01.887 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:01.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:01.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:01.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:01.989 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:01.989 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:01.989 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:01.990 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:01.990 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:01.990 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:02.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:02.456 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:02.456 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:02.457 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:03.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:03.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:03.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:03.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:03.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:03.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:03.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:03.384 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:03.384 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:03.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=18346465 2020-09-12T14:12:03.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 18346465 '!=' null 2020-09-12T14:12:03.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:03.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:03.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:04.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:04.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:04.021 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:04.022 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:04.022 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:04.022 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:04.022 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:04.022 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:04.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:04.507 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:04.507 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:04.507 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:05.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:05.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:05.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:05.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:05.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:05.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:05.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:05.081 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:05.081 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:05.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=23588708 2020-09-12T14:12:05.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 23588708 '!=' null 2020-09-12T14:12:05.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:05.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:05.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:06.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:06.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:06.096 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:06.097 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:06.097 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:06.097 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:06.097 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:06.098 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:06.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:06.512 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:06.512 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:06.513 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:07.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:07.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:07.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:07.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:07.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:07.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:07.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:07.168 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:07.169 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:07.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=20967756 2020-09-12T14:12:07.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 20967756 '!=' null 2020-09-12T14:12:07.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:07.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:07.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:07.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:07.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:07.812 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:07.812 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:07.813 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:07.813 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:07.813 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:07.813 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:08.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:08.221 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:08.221 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:08.222 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:08.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:08.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:08.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:08.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:08.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:08.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:08.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:08.756 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:08.756 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:09.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=22714902 2020-09-12T14:12:09.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 22714902 '!=' null 2020-09-12T14:12:09.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:09.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:09.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:09.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:09.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:09.390 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:09.390 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:09.390 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:09.390 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:09.391 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:09.391 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:09.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:09.799 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:09.800 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:09.800 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:10.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:10.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:10.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:10.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:10.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:10.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:10.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:10.406 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:10.406 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:10.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=22715012 2020-09-12T14:12:10.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 22715012 '!=' null 2020-09-12T14:12:10.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:10.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:10.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:11.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:11.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:11.062 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:11.062 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:11.062 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:11.063 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:11.063 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:11.063 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:11.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:11.519 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:11.520 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:11.520 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:12.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:12.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:12.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:12.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:12.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:12.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:12.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:12.362 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:12.362 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:12.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=17473136 2020-09-12T14:12:12.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 17473136 '!=' null 2020-09-12T14:12:12.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:12.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:12.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:13.079 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:13.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:13.080 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:13.081 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:13.081 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:13.081 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:13.081 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:13.081 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:13.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:13.479 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:13.480 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:13.480 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:14.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:14.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:14.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:14.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:14.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:14.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:14.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:14.027 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:14.027 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:14.673 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=21841409 2020-09-12T14:12:14.673 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 21841409 '!=' null 2020-09-12T14:12:14.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:14.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:14.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:14.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:14.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:14.775 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:14.775 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:14.775 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:14.775 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:14.776 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:14.776 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:15.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:15.172 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:15.173 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:15.173 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:15.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:15.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:15.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:15.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:15.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:15.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:15.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:15.725 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:15.725 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:16.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=21841416 2020-09-12T14:12:16.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 21841416 '!=' null 2020-09-12T14:12:16.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:16.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:16.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:16.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:16.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:16.499 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:16.500 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:16.500 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:16.500 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:16.500 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:16.501 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:16.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:16.899 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:16.900 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:16.900 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:17.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:17.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:17.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:17.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:17.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:17.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:17.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:17.475 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:17.476 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:18.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=21841416 2020-09-12T14:12:18.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 21841416 '!=' null 2020-09-12T14:12:18.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:18.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:18.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:18.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:18.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:18.485 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:18.485 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:18.485 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:18.485 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:18.486 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:18.486 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:18.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:18.932 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:18.932 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:18.933 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:19.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:19.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:19.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:19.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:19.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:19.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:19.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:19.489 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:19.489 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:20.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=21841562 2020-09-12T14:12:20.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 21841562 '!=' null 2020-09-12T14:12:20.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:20.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:20.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:20.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:20.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:20.128 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:20.128 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:20.129 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:20.129 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:20.129 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:20.129 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:20.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:20.578 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:20.579 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:20.579 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:21.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:21.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:21.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:21.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:21.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:21.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:21.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:21.437 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:21.437 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:22.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=22715443 2020-09-12T14:12:22.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 22715443 '!=' null 2020-09-12T14:12:22.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:22.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:22.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:22.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:22.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:22.202 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:22.202 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:22.202 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:22.202 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:22.202 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:22.202 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:22.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:22.601 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:22.601 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:22.601 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:23.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:23.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:23.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:23.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:23.172 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:23.172 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:23.172 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:23.172 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:23.172 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:23.762 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=19220684 2020-09-12T14:12:23.762 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 19220684 '!=' null 2020-09-12T14:12:23.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:23.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:23.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:23.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:23.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:23.864 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:23.864 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:23.864 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:23.865 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:23.865 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:23.865 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:24.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:24.368 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:24.369 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:24.369 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:24.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:24.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:24.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:24.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:24.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:24.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:24.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:24.961 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:24.961 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:25.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=22715296 2020-09-12T14:12:25.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 22715296 '!=' null 2020-09-12T14:12:25.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:25.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:25.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:25.662 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:25.662 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:25.663 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:25.663 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:25.663 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:25.664 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:25.664 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:25.664 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:26.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:26.122 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:26.122 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:26.122 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:26.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:26.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:26.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:26.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:26.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:26.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:26.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:26.875 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:26.875 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:27.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=22715353 2020-09-12T14:12:27.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 22715353 '!=' null 2020-09-12T14:12:27.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:27.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:27.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:27.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:27.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:27.982 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:27.982 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:27.983 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:27.983 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:27.983 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:27.983 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:28.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:28.465 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:28.466 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:28.466 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:29.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:29.106 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:29.106 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:29.106 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:29.106 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:29.106 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:29.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:29.107 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:29.107 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:29.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=22715350 2020-09-12T14:12:29.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 22715350 '!=' null 2020-09-12T14:12:29.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:29.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:29.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:30.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:30.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:30.040 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:30.041 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:30.041 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:30.041 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:30.041 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:30.041 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:30.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:30.442 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:30.443 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:30.443 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:30.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:31.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:31.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:31.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:31.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:31.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:31.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:31.001 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:31.001 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:31.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=21841683 2020-09-12T14:12:31.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 21841683 '!=' null 2020-09-12T14:12:31.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:31.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:31.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:31.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:31.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:31.787 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:31.787 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:31.787 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:31.788 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:31.788 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:31.788 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:32.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:32.195 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:32.196 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:32.196 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:32.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:32.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:32.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:32.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:32.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:32.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:32.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:32.982 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:32.982 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:34.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=17473333 2020-09-12T14:12:34.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 17473333 '!=' null 2020-09-12T14:12:34.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:34.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:34.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:34.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:34.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:34.150 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:34.150 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:34.151 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:34.151 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:34.151 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:34.151 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:34.547 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:34.548 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:34.548 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:34.548 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:35.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:35.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:35.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:35.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:35.264 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:35.264 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:35.264 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:35.264 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:35.265 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:36.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=20094403 2020-09-12T14:12:36.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 20094403 '!=' null 2020-09-12T14:12:36.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:36.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:36.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:36.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:36.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:36.170 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:36.171 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:36.171 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:36.171 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:36.171 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:36.171 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:36.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:36.575 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:36.576 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:36.576 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:37.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:37.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:37.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:37.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:37.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:37.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:37.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:37.234 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:37.235 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:37.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=18347036 2020-09-12T14:12:37.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 18347036 '!=' null 2020-09-12T14:12:37.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:37.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:37.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:37.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:37.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:37.959 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:37.959 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:37.959 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:37.959 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:37.959 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:37.959 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:38.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:38.441 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:38.441 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:38.442 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:39.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:39.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:39.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:39.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:39.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:39.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:39.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:39.185 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:39.185 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:39.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=16599579 2020-09-12T14:12:39.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 16599579 '!=' null 2020-09-12T14:12:39.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:39.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:39.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:39.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:39.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:39.870 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:39.870 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:39.870 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:39.870 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:39.870 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:39.870 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:40.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:40.294 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:40.294 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:40.294 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:40.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:40.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:40.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:40.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:40.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:40.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:40.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:40.829 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:40.829 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:41.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=12580611 2020-09-12T14:12:41.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 12580611 '!=' null 2020-09-12T14:12:41.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:41.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:41.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:41.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:41.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:41.476 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:41.476 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:41.476 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:41.477 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:41.477 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:41.477 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:41.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:41.878 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:41.878 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:41.879 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:42.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:42.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:42.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:42.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:42.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:42.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:42.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:42.426 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:42.426 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:42.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=10483979 2020-09-12T14:12:42.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 10483979 '!=' null 2020-09-12T14:12:42.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:42.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:42.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:43.083 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:43.083 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:43.084 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:43.084 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:43.084 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:43.084 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:43.085 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:43.085 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:43.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:12:43.479 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:43.479 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:43.480 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:44.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 2 2020-09-12T14:12:44.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:12:44.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:12:44.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:12:44.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:12:44.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:12:44.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:12:44.016 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:12:44.017 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:12:44.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=6815993 2020-09-12T14:12:44.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 6815993 '!=' null 2020-09-12T14:12:44.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:12:44.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:44.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:44.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:44.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:44.682 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:44.682 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:44.682 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:44.683 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:44.683 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:44.683 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:45.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:12:45.076 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:45.076 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:45.076 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:45.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 2 2020-09-12T14:12:45.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 1 '!=' 0 2020-09-12T14:12:45.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1598: wait_for_clean: loop=0 2020-09-12T14:12:45.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1599: wait_for_clean: num_active_clean=1 2020-09-12T14:12:45.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:12:45.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:12:45.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:12:45.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:12:45.709 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:12:45.710 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:12:45.710 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:12:45.710 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:12:45.710 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:12:45.710 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:12:46.106 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=2 2020-09-12T14:12:46.107 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:12:46.107 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:12:46.107 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:12:46.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 2 = 2 2020-09-12T14:12:46.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T14:12:46.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T14:12:46.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:502: TEST_backfill_pool_priority: '[' 0 '!=' 0 ']' 2020-09-12T14:12:46.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:506: TEST_backfill_pool_priority: echo TEST PASSED 2020-09-12T14:12:46.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:509: TEST_backfill_pool_priority: delete_pool test1 2020-09-12T14:12:46.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: delete_pool: local poolname=test1 2020-09-12T14:12:46.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: ceph osd pool delete test1 test1 --yes-i-really-really-mean-it 2020-09-12T14:12:46.642 INFO:tasks.workunit.client.0.smithi027.stdout:TEST PASSED 2020-09-12T14:12:47.300 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test1' does not exist 2020-09-12T14:12:47.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:510: TEST_backfill_pool_priority: delete_pool test2 2020-09-12T14:12:47.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: delete_pool: local poolname=test2 2020-09-12T14:12:47.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: ceph osd pool delete test2 test2 --yes-i-really-really-mean-it 2020-09-12T14:12:47.840 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test2' does not exist 2020-09-12T14:12:47.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:511: TEST_backfill_pool_priority: kill_daemons td/osd-backfill-prio 2020-09-12T14:12:47.851 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:12:47.851 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:12:47.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:12:47.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:12:47.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:12:47.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:12:47.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:512: TEST_backfill_pool_priority: return 0 2020-09-12T14:12:47.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:40: run: teardown td/osd-backfill-prio 2020-09-12T14:12:47.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-backfill-prio 2020-09-12T14:12:47.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:12:47.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-backfill-prio KILL 2020-09-12T14:12:47.960 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:12:47.960 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:12:47.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:12:47.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:12:47.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:12:47.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:12:47.963 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:12:47.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:12:47.964 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:12:47.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:12:47.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:12:47.965 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:12:47.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:12:47.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:12:47.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:12:47.968 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:12:47.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:12:47.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:12:47.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-backfill-prio 2020-09-12T14:12:47.981 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:12:47.982 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:12:47.982 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:12:47.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.73258 2020-09-12T14:12:47.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:12:47.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:12:47.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:37: run: for func in $funcs 2020-09-12T14:12:47.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:38: run: setup td/osd-backfill-prio 2020-09-12T14:12:47.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-backfill-prio 2020-09-12T14:12:47.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-backfill-prio 2020-09-12T14:12:47.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-backfill-prio 2020-09-12T14:12:47.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:12:47.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-backfill-prio KILL 2020-09-12T14:12:47.984 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:12:47.985 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:12:47.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:12:47.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:12:47.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:12:47.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:12:47.986 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:12:47.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:12:47.986 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:12:47.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:12:47.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:12:47.987 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:12:47.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:12:47.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:12:47.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:12:47.989 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:12:47.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:12:47.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:12:47.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-backfill-prio 2020-09-12T14:12:47.992 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:12:47.992 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:12:47.992 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:12:47.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.73258 2020-09-12T14:12:47.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:12:47.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:12:47.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-backfill-prio 2020-09-12T14:12:47.994 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2020-09-12T14:12:47.994 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:12:47.994 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:12:47.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.73258 2020-09-12T14:12:47.995 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2020-09-12T14:12:47.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2020-09-12T14:12:47.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2020-09-12T14:12:47.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-backfill-prio 1' TERM HUP INT 2020-09-12T14:12:47.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:39: run: TEST_backfill_priority td/osd-backfill-prio 2020-09-12T14:12:47.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:46: TEST_backfill_priority: local dir=td/osd-backfill-prio 2020-09-12T14:12:47.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:47: TEST_backfill_priority: local pools=10 2020-09-12T14:12:47.997 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:48: TEST_backfill_priority: local OSDS=5 2020-09-12T14:12:47.997 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:50: TEST_backfill_priority: expr 150 + 1 2020-09-12T14:12:47.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:50: TEST_backfill_priority: local degraded_prio=151 2020-09-12T14:12:47.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:51: TEST_backfill_priority: local max_tries=10 2020-09-12T14:12:47.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:53: TEST_backfill_priority: run_mon td/osd-backfill-prio a 2020-09-12T14:12:47.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-backfill-prio 2020-09-12T14:12:47.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2020-09-12T14:12:47.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2020-09-12T14:12:47.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2020-09-12T14:12:47.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-backfill-prio/a 2020-09-12T14:12:47.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-backfill-prio/a --run-dir=td/osd-backfill-prio 2020-09-12T14:12:48.306 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2020-09-12T14:12:48.306 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:12:48.306 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:12:48.306 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:12:48.306 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:12:48.307 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:12:48.307 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.73258/$cluster-$name.asok' 2020-09-12T14:12:48.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-backfill-prio/a '--log-file=td/osd-backfill-prio/$name.log' '--admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' --mon-cluster-log-file=td/osd-backfill-prio/log --run-dir=td/osd-backfill-prio '--pid-file=td/osd-backfill-prio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 2020-09-12T14:12:48.482 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: cat 2020-09-12T14:12:48.483 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a fsid 2020-09-12T14:12:48.483 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:12:48.483 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:12:48.484 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=fsid 2020-09-12T14:12:48.484 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .fsid 2020-09-12T14:12:48.484 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:12:48.484 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:12:48.484 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:12:48.484 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:12:48.485 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:12:48.485 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:12:48.485 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.73258/ceph-mon.a.asok 2020-09-12T14:12:48.485 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:12:48.486 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.73258/ceph-mon.a.asok config get fsid 2020-09-12T14:12:48.587 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a mon_host 2020-09-12T14:12:48.587 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:12:48.588 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:12:48.588 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=mon_host 2020-09-12T14:12:48.588 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .mon_host 2020-09-12T14:12:48.588 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:12:48.588 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:12:48.588 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:12:48.589 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:12:48.589 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:12:48.590 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:12:48.590 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.73258/ceph-mon.a.asok 2020-09-12T14:12:48.590 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:12:48.590 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.73258/ceph-mon.a.asok config get mon_host 2020-09-12T14:12:48.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:54: TEST_backfill_priority: run_mgr td/osd-backfill-prio x 2020-09-12T14:12:48.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: local dir=td/osd-backfill-prio 2020-09-12T14:12:48.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: shift 2020-09-12T14:12:48.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: local id=x 2020-09-12T14:12:48.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: shift 2020-09-12T14:12:48.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: local data=td/osd-backfill-prio/x 2020-09-12T14:12:48.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:558: run_mgr: ceph config set mgr mgr/devicehealth/enable_monitoring off --force 2020-09-12T14:12:49.596 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: get_asok_path 2020-09-12T14:12:49.596 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:12:49.597 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:12:49.597 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:12:49.597 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:12:49.597 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:12:49.597 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.73258/$cluster-$name.asok' 2020-09-12T14:12:49.598 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:12:49.598 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-backfill-prio/x '--log-file=td/osd-backfill-prio/$name.log' '--admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' --run-dir=td/osd-backfill-prio '--pid-file=td/osd-backfill-prio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:12:49.622 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:55: TEST_backfill_priority: export CEPH_ARGS 2020-09-12T14:12:49.623 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:57: TEST_backfill_priority: expr 5 - 1 2020-09-12T14:12:49.624 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:57: TEST_backfill_priority: seq 0 4 2020-09-12T14:12:49.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:57: TEST_backfill_priority: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:12:49.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:59: TEST_backfill_priority: run_osd td/osd-backfill-prio 0 2020-09-12T14:12:49.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-prio 2020-09-12T14:12:49.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:12:49.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=0 2020-09-12T14:12:49.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:12:49.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-prio/0 2020-09-12T14:12:49.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=576afb9f-26d0-41c8-a49c-302fc5ab0911 --auth-supported=none --mon-host=127.0.0.1:7114 --osd_max_backfills=1 --debug_reserver=20 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 ' 2020-09-12T14:12:49.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:12:49.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:12:49.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:12:49.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-prio/0' 2020-09-12T14:12:49.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-prio/0/journal' 2020-09-12T14:12:49.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:12:49.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:12:49.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-prio' 2020-09-12T14:12:49.634 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:12:49.634 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:12:49.634 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:12:49.634 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:12:49.635 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:12:49.635 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:12:49.635 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.73258/$cluster-$name.asok' 2020-09-12T14:12:49.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' 2020-09-12T14:12:49.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:12:49.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:12:49.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:12:49.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-prio/$name.log' 2020-09-12T14:12:49.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-prio/$name.pid' 2020-09-12T14:12:49.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:12:49.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:12:49.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:12:49.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:12:49.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:12:49.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-prio/0 2020-09-12T14:12:49.639 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:12:49.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=d5c918ae-a5bc-4ec3-a88a-3cb37dd8e507 2020-09-12T14:12:49.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd0 d5c918ae-a5bc-4ec3-a88a-3cb37dd8e507' 2020-09-12T14:12:49.640 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:12:49.640 INFO:tasks.workunit.client.0.smithi027.stdout:add osd0 d5c918ae-a5bc-4ec3-a88a-3cb37dd8e507 2020-09-12T14:12:49.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQBh11xfAhMTJxAA8ztnkRCTQRm5JyPAW0rXwg== 2020-09-12T14:12:49.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQBh11xfAhMTJxAA8ztnkRCTQRm5JyPAW0rXwg=="}' 2020-09-12T14:12:49.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new d5c918ae-a5bc-4ec3-a88a-3cb37dd8e507 -i td/osd-backfill-prio/0/new.json 2020-09-12T14:12:49.986 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:12:49.997 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-prio/0/new.json 2020-09-12T14:12:49.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 0 --fsid=576afb9f-26d0-41c8-a49c-302fc5ab0911 --auth-supported=none --mon-host=127.0.0.1:7114 --osd_max_backfills=1 --debug_reserver=20 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-prio/0 --osd-journal=td/osd-backfill-prio/0/journal --chdir= --run-dir=td/osd-backfill-prio '--admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-prio/$name.log' '--pid-file=td/osd-backfill-prio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQBh11xfAhMTJxAA8ztnkRCTQRm5JyPAW0rXwg== --osd-uuid d5c918ae-a5bc-4ec3-a88a-3cb37dd8e507 2020-09-12T14:12:50.025 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:12:50.024+0000 7fa52a658ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:12:50.027 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:12:50.026+0000 7fa52a658ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:12:50.028 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:12:50.027+0000 7fa52a658ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:12:50.029 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:12:50.028+0000 7fa52a658ec0 -1 bluestore(td/osd-backfill-prio/0/block) _read_bdev_label failed to open td/osd-backfill-prio/0/block: (2) No such file or directory 2020-09-12T14:12:50.029 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:12:50.028+0000 7fa52a658ec0 -1 bluestore(td/osd-backfill-prio/0/block) _read_bdev_label failed to open td/osd-backfill-prio/0/block: (2) No such file or directory 2020-09-12T14:12:50.029 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:12:50.028+0000 7fa52a658ec0 -1 bluestore(td/osd-backfill-prio/0/block) _read_bdev_label failed to open td/osd-backfill-prio/0/block: (2) No such file or directory 2020-09-12T14:12:50.093 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:12:50.091+0000 7fa52a658ec0 -1 bluestore(td/osd-backfill-prio/0) _read_fsid unparsable uuid 2020-09-12T14:12:54.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-prio/0/keyring 2020-09-12T14:12:54.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:12:54.647 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd0 key to auth repository 2020-09-12T14:12:54.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd0 key to auth repository 2020-09-12T14:12:54.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-prio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:12:55.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.0 2020-09-12T14:12:55.189 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.0 2020-09-12T14:12:55.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:12:55.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 0 --fsid=576afb9f-26d0-41c8-a49c-302fc5ab0911 --auth-supported=none --mon-host=127.0.0.1:7114 --osd_max_backfills=1 --debug_reserver=20 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-prio/0 --osd-journal=td/osd-backfill-prio/0/journal --chdir= --run-dir=td/osd-backfill-prio '--admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-prio/$name.log' '--pid-file=td/osd-backfill-prio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:12:55.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:12:55.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:12:55.214 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:12:55.213+0000 7f98c1854ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:12:55.246 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:12:55.244+0000 7f98c1854ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:12:55.247 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:12:55.246+0000 7f98c1854ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:12:55.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 0 2020-09-12T14:12:55.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:12:55.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T14:12:55.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:12:55.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:12:55.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:12:55.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:12:55.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:12:55.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:12:55.603 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:12:55.768 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:12:55.767+0000 7f98c1854ec0 -1 Falling back to public interface 2020-09-12T14:12:56.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:12:56.603 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:12:56.601+0000 7f98c1854ec0 -1 osd.0 0 log_to_monitors {default=true} 2020-09-12T14:12:57.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:12:57.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:12:57.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:12:57.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:12:57.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:12:57.077 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:12:57.539 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/82208,v1:127.0.0.1:6803/82208] [v2:127.0.0.1:6804/82208,v1:127.0.0.1:6805/82208] exists,up d5c918ae-a5bc-4ec3-a88a-3cb37dd8e507 2020-09-12T14:12:57.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:12:57.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:12:57.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:12:57.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:57: TEST_backfill_priority: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:12:57.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:59: TEST_backfill_priority: run_osd td/osd-backfill-prio 1 2020-09-12T14:12:57.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-prio 2020-09-12T14:12:57.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:12:57.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=1 2020-09-12T14:12:57.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:12:57.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-prio/1 2020-09-12T14:12:57.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=576afb9f-26d0-41c8-a49c-302fc5ab0911 --auth-supported=none --mon-host=127.0.0.1:7114 --osd_max_backfills=1 --debug_reserver=20 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 ' 2020-09-12T14:12:57.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:12:57.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:12:57.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:12:57.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-prio/1' 2020-09-12T14:12:57.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-prio/1/journal' 2020-09-12T14:12:57.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:12:57.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:12:57.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-prio' 2020-09-12T14:12:57.542 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:12:57.542 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:12:57.542 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:12:57.543 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:12:57.543 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:12:57.543 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:12:57.543 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.73258/$cluster-$name.asok' 2020-09-12T14:12:57.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' 2020-09-12T14:12:57.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:12:57.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:12:57.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:12:57.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-prio/$name.log' 2020-09-12T14:12:57.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-prio/$name.pid' 2020-09-12T14:12:57.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:12:57.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:12:57.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:12:57.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:12:57.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:12:57.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-prio/1 2020-09-12T14:12:57.545 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:12:57.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=87d6ab06-6dd3-4d0a-becc-0c6ad126122b 2020-09-12T14:12:57.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd1 87d6ab06-6dd3-4d0a-becc-0c6ad126122b' 2020-09-12T14:12:57.546 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:12:57.546 INFO:tasks.workunit.client.0.smithi027.stdout:add osd1 87d6ab06-6dd3-4d0a-becc-0c6ad126122b 2020-09-12T14:12:57.557 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQBp11xfe6I1IRAAlqTXLSlA3p03u7EeO/uQUQ== 2020-09-12T14:12:57.557 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQBp11xfe6I1IRAAlqTXLSlA3p03u7EeO/uQUQ=="}' 2020-09-12T14:12:57.558 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 87d6ab06-6dd3-4d0a-becc-0c6ad126122b -i td/osd-backfill-prio/1/new.json 2020-09-12T14:12:58.085 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:12:58.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-prio/1/new.json 2020-09-12T14:12:58.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 1 --fsid=576afb9f-26d0-41c8-a49c-302fc5ab0911 --auth-supported=none --mon-host=127.0.0.1:7114 --osd_max_backfills=1 --debug_reserver=20 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-prio/1 --osd-journal=td/osd-backfill-prio/1/journal --chdir= --run-dir=td/osd-backfill-prio '--admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-prio/$name.log' '--pid-file=td/osd-backfill-prio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQBp11xfe6I1IRAAlqTXLSlA3p03u7EeO/uQUQ== --osd-uuid 87d6ab06-6dd3-4d0a-becc-0c6ad126122b 2020-09-12T14:12:58.126 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:12:58.125+0000 7f0f95c2eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:12:58.127 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:12:58.127+0000 7f0f95c2eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:12:58.128 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:12:58.128+0000 7f0f95c2eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:12:58.128 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:12:58.128+0000 7f0f95c2eec0 -1 bluestore(td/osd-backfill-prio/1/block) _read_bdev_label failed to open td/osd-backfill-prio/1/block: (2) No such file or directory 2020-09-12T14:12:58.129 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:12:58.128+0000 7f0f95c2eec0 -1 bluestore(td/osd-backfill-prio/1/block) _read_bdev_label failed to open td/osd-backfill-prio/1/block: (2) No such file or directory 2020-09-12T14:12:58.129 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:12:58.128+0000 7f0f95c2eec0 -1 bluestore(td/osd-backfill-prio/1/block) _read_bdev_label failed to open td/osd-backfill-prio/1/block: (2) No such file or directory 2020-09-12T14:12:58.170 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:12:58.169+0000 7f0f95c2eec0 -1 bluestore(td/osd-backfill-prio/1) _read_fsid unparsable uuid 2020-09-12T14:13:02.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-prio/1/keyring 2020-09-12T14:13:02.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:13:02.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd1 key to auth repository 2020-09-12T14:13:02.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-prio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:13:02.494 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd1 key to auth repository 2020-09-12T14:13:03.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.1 2020-09-12T14:13:03.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 1 --fsid=576afb9f-26d0-41c8-a49c-302fc5ab0911 --auth-supported=none --mon-host=127.0.0.1:7114 --osd_max_backfills=1 --debug_reserver=20 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-prio/1 --osd-journal=td/osd-backfill-prio/1/journal --chdir= --run-dir=td/osd-backfill-prio '--admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-prio/$name.log' '--pid-file=td/osd-backfill-prio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:13:03.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:13:03.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:13:03.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:13:03.046 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T14:13:03.070 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:03.070+0000 7ff234b99ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:13:03.072 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:03.072+0000 7ff234b99ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:13:03.074 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:03.073+0000 7ff234b99ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:13:03.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 1 2020-09-12T14:13:03.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:13:03.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:13:03.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:13:03.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:13:03.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:13:03.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:13:03.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:13:03.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:13:03.465 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:13:03.592 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:03.591+0000 7ff234b99ec0 -1 Falling back to public interface 2020-09-12T14:13:03.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:13:04.410 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:04.409+0000 7ff234b99ec0 -1 osd.1 0 log_to_monitors {default=true} 2020-09-12T14:13:04.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:13:04.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:13:04.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:13:04.891 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:13:04.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:13:04.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:13:05.333 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/83202,v1:127.0.0.1:6811/83202] [v2:127.0.0.1:6812/83202,v1:127.0.0.1:6813/83202] exists,up 87d6ab06-6dd3-4d0a-becc-0c6ad126122b 2020-09-12T14:13:05.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:13:05.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:13:05.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:13:05.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:57: TEST_backfill_priority: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:13:05.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:59: TEST_backfill_priority: run_osd td/osd-backfill-prio 2 2020-09-12T14:13:05.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-prio 2020-09-12T14:13:05.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:13:05.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=2 2020-09-12T14:13:05.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:13:05.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-prio/2 2020-09-12T14:13:05.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=576afb9f-26d0-41c8-a49c-302fc5ab0911 --auth-supported=none --mon-host=127.0.0.1:7114 --osd_max_backfills=1 --debug_reserver=20 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 ' 2020-09-12T14:13:05.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:13:05.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:13:05.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:13:05.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-prio/2' 2020-09-12T14:13:05.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-prio/2/journal' 2020-09-12T14:13:05.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:13:05.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:13:05.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-prio' 2020-09-12T14:13:05.336 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:13:05.336 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:13:05.336 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:13:05.336 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:13:05.337 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:13:05.337 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:13:05.337 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.73258/$cluster-$name.asok' 2020-09-12T14:13:05.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' 2020-09-12T14:13:05.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:13:05.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:13:05.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:13:05.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-prio/$name.log' 2020-09-12T14:13:05.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-prio/$name.pid' 2020-09-12T14:13:05.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:13:05.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:13:05.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:13:05.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:13:05.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:13:05.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-prio/2 2020-09-12T14:13:05.339 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:13:05.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=26a4075e-2f98-4913-992e-2f9c3881470b 2020-09-12T14:13:05.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd2 26a4075e-2f98-4913-992e-2f9c3881470b' 2020-09-12T14:13:05.340 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:13:05.340 INFO:tasks.workunit.client.0.smithi027.stdout:add osd2 26a4075e-2f98-4913-992e-2f9c3881470b 2020-09-12T14:13:05.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQBx11xfGFLuFBAAOptfjkoC3/xw6gxspSD7Ww== 2020-09-12T14:13:05.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQBx11xfGFLuFBAAOptfjkoC3/xw6gxspSD7Ww=="}' 2020-09-12T14:13:05.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 26a4075e-2f98-4913-992e-2f9c3881470b -i td/osd-backfill-prio/2/new.json 2020-09-12T14:13:05.996 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:13:06.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-prio/2/new.json 2020-09-12T14:13:06.009 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 2 --fsid=576afb9f-26d0-41c8-a49c-302fc5ab0911 --auth-supported=none --mon-host=127.0.0.1:7114 --osd_max_backfills=1 --debug_reserver=20 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-prio/2 --osd-journal=td/osd-backfill-prio/2/journal --chdir= --run-dir=td/osd-backfill-prio '--admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-prio/$name.log' '--pid-file=td/osd-backfill-prio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQBx11xfGFLuFBAAOptfjkoC3/xw6gxspSD7Ww== --osd-uuid 26a4075e-2f98-4913-992e-2f9c3881470b 2020-09-12T14:13:06.036 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:06.035+0000 7fabfaa72ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:13:06.037 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:06.037+0000 7fabfaa72ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:13:06.039 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:06.038+0000 7fabfaa72ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:13:06.040 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:06.039+0000 7fabfaa72ec0 -1 bluestore(td/osd-backfill-prio/2/block) _read_bdev_label failed to open td/osd-backfill-prio/2/block: (2) No such file or directory 2020-09-12T14:13:06.040 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:06.039+0000 7fabfaa72ec0 -1 bluestore(td/osd-backfill-prio/2/block) _read_bdev_label failed to open td/osd-backfill-prio/2/block: (2) No such file or directory 2020-09-12T14:13:06.040 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:06.039+0000 7fabfaa72ec0 -1 bluestore(td/osd-backfill-prio/2/block) _read_bdev_label failed to open td/osd-backfill-prio/2/block: (2) No such file or directory 2020-09-12T14:13:06.109 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:06.108+0000 7fabfaa72ec0 -1 bluestore(td/osd-backfill-prio/2) _read_fsid unparsable uuid 2020-09-12T14:13:10.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-prio/2/keyring 2020-09-12T14:13:10.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:13:10.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd2 key to auth repository 2020-09-12T14:13:10.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-prio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:13:10.693 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd2 key to auth repository 2020-09-12T14:13:11.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.2 2020-09-12T14:13:11.246 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 2 --fsid=576afb9f-26d0-41c8-a49c-302fc5ab0911 --auth-supported=none --mon-host=127.0.0.1:7114 --osd_max_backfills=1 --debug_reserver=20 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-prio/2 --osd-journal=td/osd-backfill-prio/2/journal --chdir= --run-dir=td/osd-backfill-prio '--admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-prio/$name.log' '--pid-file=td/osd-backfill-prio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:13:11.246 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:13:11.246 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.2 2020-09-12T14:13:11.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:13:11.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:13:11.271 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:11.271+0000 7fa19a8ecec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:13:11.290 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:11.289+0000 7fa19a8ecec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:13:11.292 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:11.291+0000 7fa19a8ecec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:13:11.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 2 2020-09-12T14:13:11.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:13:11.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T14:13:11.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:13:11.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:13:11.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:13:11.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:13:11.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:13:11.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:13:11.652 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:13:11.807 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:11.806+0000 7fa19a8ecec0 -1 Falling back to public interface 2020-09-12T14:13:12.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:13:12.617 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:12.616+0000 7fa19a8ecec0 -1 osd.2 0 log_to_monitors {default=true} 2020-09-12T14:13:13.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:13:13.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:13:13.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:13:13.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:13:13.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:13:13.060 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:13:13.501 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/84196,v1:127.0.0.1:6819/84196] [v2:127.0.0.1:6820/84196,v1:127.0.0.1:6821/84196] exists,up 26a4075e-2f98-4913-992e-2f9c3881470b 2020-09-12T14:13:13.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:13:13.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:13:13.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:13:13.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:57: TEST_backfill_priority: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:13:13.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:59: TEST_backfill_priority: run_osd td/osd-backfill-prio 3 2020-09-12T14:13:13.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-prio 2020-09-12T14:13:13.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:13:13.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=3 2020-09-12T14:13:13.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:13:13.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-prio/3 2020-09-12T14:13:13.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=576afb9f-26d0-41c8-a49c-302fc5ab0911 --auth-supported=none --mon-host=127.0.0.1:7114 --osd_max_backfills=1 --debug_reserver=20 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 ' 2020-09-12T14:13:13.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:13:13.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:13:13.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:13:13.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-prio/3' 2020-09-12T14:13:13.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-prio/3/journal' 2020-09-12T14:13:13.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:13:13.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:13:13.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-prio' 2020-09-12T14:13:13.504 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:13:13.504 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:13:13.504 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:13:13.505 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:13:13.505 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:13:13.505 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:13:13.505 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.73258/$cluster-$name.asok' 2020-09-12T14:13:13.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' 2020-09-12T14:13:13.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:13:13.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:13:13.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:13:13.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-prio/$name.log' 2020-09-12T14:13:13.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-prio/$name.pid' 2020-09-12T14:13:13.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:13:13.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:13:13.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:13:13.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:13:13.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:13:13.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-prio/3 2020-09-12T14:13:13.507 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:13:13.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=f94b7038-d289-4a7b-bb4d-146336bdaa00 2020-09-12T14:13:13.508 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd3 f94b7038-d289-4a7b-bb4d-146336bdaa00' 2020-09-12T14:13:13.508 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:13:13.508 INFO:tasks.workunit.client.0.smithi027.stdout:add osd3 f94b7038-d289-4a7b-bb4d-146336bdaa00 2020-09-12T14:13:13.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQB511xfzbH2HhAAycoMVfEwJwbVSeKpIh75Xg== 2020-09-12T14:13:13.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQB511xfzbH2HhAAycoMVfEwJwbVSeKpIh75Xg=="}' 2020-09-12T14:13:13.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new f94b7038-d289-4a7b-bb4d-146336bdaa00 -i td/osd-backfill-prio/3/new.json 2020-09-12T14:13:14.148 INFO:tasks.workunit.client.0.smithi027.stdout:3 2020-09-12T14:13:14.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-prio/3/new.json 2020-09-12T14:13:14.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 3 --fsid=576afb9f-26d0-41c8-a49c-302fc5ab0911 --auth-supported=none --mon-host=127.0.0.1:7114 --osd_max_backfills=1 --debug_reserver=20 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-prio/3 --osd-journal=td/osd-backfill-prio/3/journal --chdir= --run-dir=td/osd-backfill-prio '--admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-prio/$name.log' '--pid-file=td/osd-backfill-prio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQB511xfzbH2HhAAycoMVfEwJwbVSeKpIh75Xg== --osd-uuid f94b7038-d289-4a7b-bb4d-146336bdaa00 2020-09-12T14:13:14.188 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:14.187+0000 7fed1c5c3ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:13:14.189 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:14.189+0000 7fed1c5c3ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:13:14.191 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:14.190+0000 7fed1c5c3ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:13:14.192 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:14.191+0000 7fed1c5c3ec0 -1 bluestore(td/osd-backfill-prio/3/block) _read_bdev_label failed to open td/osd-backfill-prio/3/block: (2) No such file or directory 2020-09-12T14:13:14.192 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:14.191+0000 7fed1c5c3ec0 -1 bluestore(td/osd-backfill-prio/3/block) _read_bdev_label failed to open td/osd-backfill-prio/3/block: (2) No such file or directory 2020-09-12T14:13:14.192 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:14.191+0000 7fed1c5c3ec0 -1 bluestore(td/osd-backfill-prio/3/block) _read_bdev_label failed to open td/osd-backfill-prio/3/block: (2) No such file or directory 2020-09-12T14:13:14.325 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:14.324+0000 7fed1c5c3ec0 -1 bluestore(td/osd-backfill-prio/3) _read_fsid unparsable uuid 2020-09-12T14:13:18.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-prio/3/keyring 2020-09-12T14:13:18.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:13:18.687 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd3 key to auth repository 2020-09-12T14:13:18.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd3 key to auth repository 2020-09-12T14:13:18.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-prio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:13:19.236 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.3 2020-09-12T14:13:19.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.3 2020-09-12T14:13:19.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:13:19.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:13:19.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:13:19.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 3 --fsid=576afb9f-26d0-41c8-a49c-302fc5ab0911 --auth-supported=none --mon-host=127.0.0.1:7114 --osd_max_backfills=1 --debug_reserver=20 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-prio/3 --osd-journal=td/osd-backfill-prio/3/journal --chdir= --run-dir=td/osd-backfill-prio '--admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-prio/$name.log' '--pid-file=td/osd-backfill-prio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:13:19.262 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:19.261+0000 7f985c144ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:13:19.281 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:19.280+0000 7f985c144ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:13:19.282 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:19.281+0000 7f985c144ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:13:19.660 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 3 2020-09-12T14:13:19.660 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:13:19.660 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=3 2020-09-12T14:13:19.660 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:13:19.660 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:13:19.660 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:13:19.661 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:13:19.661 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:13:19.661 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.3 up' 2020-09-12T14:13:19.661 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:13:19.797 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:19.796+0000 7f985c144ec0 -1 Falling back to public interface 2020-09-12T14:13:20.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:13:20.625 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:20.623+0000 7f985c144ec0 -1 osd.3 0 log_to_monitors {default=true} 2020-09-12T14:13:21.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:13:21.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:13:21.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:13:21.075 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:13:21.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.3 up' 2020-09-12T14:13:21.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:13:21.535 INFO:tasks.workunit.client.0.smithi027.stdout:osd.3 up in weight 1 up_from 20 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/85190,v1:127.0.0.1:6827/85190] [v2:127.0.0.1:6828/85190,v1:127.0.0.1:6829/85190] exists,up f94b7038-d289-4a7b-bb4d-146336bdaa00 2020-09-12T14:13:21.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:13:21.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:13:21.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:13:21.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:57: TEST_backfill_priority: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:13:21.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:59: TEST_backfill_priority: run_osd td/osd-backfill-prio 4 2020-09-12T14:13:21.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-prio 2020-09-12T14:13:21.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:13:21.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=4 2020-09-12T14:13:21.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:13:21.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-prio/4 2020-09-12T14:13:21.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=576afb9f-26d0-41c8-a49c-302fc5ab0911 --auth-supported=none --mon-host=127.0.0.1:7114 --osd_max_backfills=1 --debug_reserver=20 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 ' 2020-09-12T14:13:21.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:13:21.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:13:21.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:13:21.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-prio/4' 2020-09-12T14:13:21.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-prio/4/journal' 2020-09-12T14:13:21.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:13:21.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:13:21.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-prio' 2020-09-12T14:13:21.538 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:13:21.538 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:13:21.538 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:13:21.539 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:13:21.539 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:13:21.539 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:13:21.539 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.73258/$cluster-$name.asok' 2020-09-12T14:13:21.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' 2020-09-12T14:13:21.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:13:21.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:13:21.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:13:21.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-prio/$name.log' 2020-09-12T14:13:21.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-prio/$name.pid' 2020-09-12T14:13:21.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:13:21.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:13:21.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:13:21.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:13:21.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:13:21.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-prio/4 2020-09-12T14:13:21.541 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:13:21.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=d207fe49-d189-421a-a7d1-76dfac34c5b4 2020-09-12T14:13:21.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd4 d207fe49-d189-421a-a7d1-76dfac34c5b4' 2020-09-12T14:13:21.542 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:13:21.542 INFO:tasks.workunit.client.0.smithi027.stdout:add osd4 d207fe49-d189-421a-a7d1-76dfac34c5b4 2020-09-12T14:13:21.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQCB11xfFl77IBAAfB+bnIJzoBifskpvKf9IWg== 2020-09-12T14:13:21.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQCB11xfFl77IBAAfB+bnIJzoBifskpvKf9IWg=="}' 2020-09-12T14:13:21.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new d207fe49-d189-421a-a7d1-76dfac34c5b4 -i td/osd-backfill-prio/4/new.json 2020-09-12T14:13:22.157 INFO:tasks.workunit.client.0.smithi027.stdout:4 2020-09-12T14:13:22.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-prio/4/new.json 2020-09-12T14:13:22.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 4 --fsid=576afb9f-26d0-41c8-a49c-302fc5ab0911 --auth-supported=none --mon-host=127.0.0.1:7114 --osd_max_backfills=1 --debug_reserver=20 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-prio/4 --osd-journal=td/osd-backfill-prio/4/journal --chdir= --run-dir=td/osd-backfill-prio '--admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-prio/$name.log' '--pid-file=td/osd-backfill-prio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQCB11xfFl77IBAAfB+bnIJzoBifskpvKf9IWg== --osd-uuid d207fe49-d189-421a-a7d1-76dfac34c5b4 2020-09-12T14:13:22.196 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:22.195+0000 7fc61fa5eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:13:22.341 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:22.339+0000 7fc61fa5eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:13:22.342 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:22.340+0000 7fc61fa5eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:13:22.342 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:22.341+0000 7fc61fa5eec0 -1 bluestore(td/osd-backfill-prio/4/block) _read_bdev_label failed to open td/osd-backfill-prio/4/block: (2) No such file or directory 2020-09-12T14:13:22.342 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:22.341+0000 7fc61fa5eec0 -1 bluestore(td/osd-backfill-prio/4/block) _read_bdev_label failed to open td/osd-backfill-prio/4/block: (2) No such file or directory 2020-09-12T14:13:22.343 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:22.341+0000 7fc61fa5eec0 -1 bluestore(td/osd-backfill-prio/4/block) _read_bdev_label failed to open td/osd-backfill-prio/4/block: (2) No such file or directory 2020-09-12T14:13:22.449 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:22.447+0000 7fc61fa5eec0 -1 bluestore(td/osd-backfill-prio/4) _read_fsid unparsable uuid 2020-09-12T14:13:26.997 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-prio/4/keyring 2020-09-12T14:13:26.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:13:26.999 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd4 key to auth repository 2020-09-12T14:13:26.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd4 key to auth repository 2020-09-12T14:13:26.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-prio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:13:27.542 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.4 2020-09-12T14:13:27.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.4 2020-09-12T14:13:27.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 4 --fsid=576afb9f-26d0-41c8-a49c-302fc5ab0911 --auth-supported=none --mon-host=127.0.0.1:7114 --osd_max_backfills=1 --debug_reserver=20 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-prio/4 --osd-journal=td/osd-backfill-prio/4/journal --chdir= --run-dir=td/osd-backfill-prio '--admin-socket=/tmp/ceph-asok.73258/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-prio/$name.log' '--pid-file=td/osd-backfill-prio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:13:27.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:13:27.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:13:27.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:13:27.568 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:27.566+0000 7fcda2386ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:13:27.569 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:27.568+0000 7fcda2386ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:13:27.570 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:27.569+0000 7fcda2386ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:13:27.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 4 2020-09-12T14:13:27.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:13:27.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=4 2020-09-12T14:13:27.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:13:27.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:13:27.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:13:27.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:13:27.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:13:27.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.4 up' 2020-09-12T14:13:27.952 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:13:28.086 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:28.084+0000 7fcda2386ec0 -1 Falling back to public interface 2020-09-12T14:13:28.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:13:29.018 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:13:29.016+0000 7fcda2386ec0 -1 osd.4 0 log_to_monitors {default=true} 2020-09-12T14:13:29.378 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:13:29.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:13:29.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:13:29.379 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:13:29.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:13:29.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.4 up' 2020-09-12T14:13:29.900 INFO:tasks.workunit.client.0.smithi027.stdout:osd.4 up in weight 1 up_from 25 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/86184,v1:127.0.0.1:6835/86184] [v2:127.0.0.1:6836/86184,v1:127.0.0.1:6837/86184] exists,up d207fe49-d189-421a-a7d1-76dfac34c5b4 2020-09-12T14:13:29.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:13:29.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:13:29.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:13:29.901 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:62: TEST_backfill_priority: seq 1 10 2020-09-12T14:13:29.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:62: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:13:29.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:64: TEST_backfill_priority: create_pool test1 1 1 2020-09-12T14:13:29.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test1 1 1 2020-09-12T14:13:30.465 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test1' already exists 2020-09-12T14:13:30.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:13:31.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:65: TEST_backfill_priority: ceph osd pool set test1 size 2 2020-09-12T14:13:32.442 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 1 size to 2 2020-09-12T14:13:32.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:62: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:13:32.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:64: TEST_backfill_priority: create_pool test2 1 1 2020-09-12T14:13:32.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test2 1 1 2020-09-12T14:13:33.197 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test2' already exists 2020-09-12T14:13:33.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:13:34.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:65: TEST_backfill_priority: ceph osd pool set test2 size 2 2020-09-12T14:13:34.980 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 2 size to 2 2020-09-12T14:13:34.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:62: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:13:34.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:64: TEST_backfill_priority: create_pool test3 1 1 2020-09-12T14:13:34.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test3 1 1 2020-09-12T14:13:35.851 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test3' already exists 2020-09-12T14:13:35.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:13:36.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:65: TEST_backfill_priority: ceph osd pool set test3 size 2 2020-09-12T14:13:38.161 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 3 size to 2 2020-09-12T14:13:38.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:62: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:13:38.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:64: TEST_backfill_priority: create_pool test4 1 1 2020-09-12T14:13:38.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test4 1 1 2020-09-12T14:13:38.739 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test4' already exists 2020-09-12T14:13:38.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:13:39.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:65: TEST_backfill_priority: ceph osd pool set test4 size 2 2020-09-12T14:13:40.868 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 4 size to 2 2020-09-12T14:13:40.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:62: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:13:40.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:64: TEST_backfill_priority: create_pool test5 1 1 2020-09-12T14:13:40.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test5 1 1 2020-09-12T14:13:41.595 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test5' already exists 2020-09-12T14:13:41.604 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:13:42.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:65: TEST_backfill_priority: ceph osd pool set test5 size 2 2020-09-12T14:13:43.455 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 5 size to 2 2020-09-12T14:13:43.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:62: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:13:43.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:64: TEST_backfill_priority: create_pool test6 1 1 2020-09-12T14:13:43.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test6 1 1 2020-09-12T14:13:44.028 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test6' already exists 2020-09-12T14:13:44.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:13:45.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:65: TEST_backfill_priority: ceph osd pool set test6 size 2 2020-09-12T14:13:46.100 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 6 size to 2 2020-09-12T14:13:46.110 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:62: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:13:46.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:64: TEST_backfill_priority: create_pool test7 1 1 2020-09-12T14:13:46.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test7 1 1 2020-09-12T14:13:46.828 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test7' already exists 2020-09-12T14:13:46.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:13:47.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:65: TEST_backfill_priority: ceph osd pool set test7 size 2 2020-09-12T14:13:48.646 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 7 size to 2 2020-09-12T14:13:48.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:62: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:13:48.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:64: TEST_backfill_priority: create_pool test8 1 1 2020-09-12T14:13:48.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test8 1 1 2020-09-12T14:13:49.247 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test8' already exists 2020-09-12T14:13:49.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:13:50.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:65: TEST_backfill_priority: ceph osd pool set test8 size 2 2020-09-12T14:13:50.998 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 8 size to 2 2020-09-12T14:13:51.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:62: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:13:51.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:64: TEST_backfill_priority: create_pool test9 1 1 2020-09-12T14:13:51.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test9 1 1 2020-09-12T14:13:51.896 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test9' already exists 2020-09-12T14:13:51.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:13:52.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:65: TEST_backfill_priority: ceph osd pool set test9 size 2 2020-09-12T14:13:53.990 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 9 size to 2 2020-09-12T14:13:54.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:62: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:13:54.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:64: TEST_backfill_priority: create_pool test10 1 1 2020-09-12T14:13:54.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test10 1 1 2020-09-12T14:13:54.609 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test10' already exists 2020-09-12T14:13:54.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:13:55.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:65: TEST_backfill_priority: ceph osd pool set test10 size 2 2020-09-12T14:13:56.362 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 10 size to 2 2020-09-12T14:13:56.371 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:67: TEST_backfill_priority: sleep 5 2020-09-12T14:14:01.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:69: TEST_backfill_priority: wait_for_clean 2020-09-12T14:14:01.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:14:01.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:14:01.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:14:01.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:14:01.374 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 90 .1 2020-09-12T14:14:01.375 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:14:01.375 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:14:01.375 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:14:01.376 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:14:01.376 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:14:01.482 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:14:01.482 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:14:01.482 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:14:01.482 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:14:01.482 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:14:01.892 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:14:01.893 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:14:01.893 INFO:tasks.workunit.client.0.smithi027.stderr:2 2020-09-12T14:14:01.893 INFO:tasks.workunit.client.0.smithi027.stderr:3 2020-09-12T14:14:01.893 INFO:tasks.workunit.client.0.smithi027.stderr:4' 2020-09-12T14:14:01.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:14:01.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:14:01.894 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:14:02.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836495 2020-09-12T14:14:02.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836495 2020-09-12T14:14:02.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836495' 2020-09-12T14:14:02.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:14:02.037 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:14:02.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672975 2020-09-12T14:14:02.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672975 2020-09-12T14:14:02.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836495 1-42949672975' 2020-09-12T14:14:02.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:14:02.180 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:14:02.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509452 2020-09-12T14:14:02.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509452 2020-09-12T14:14:02.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836495 1-42949672975 2-64424509452' 2020-09-12T14:14:02.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:14:02.324 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2020-09-12T14:14:02.466 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=85899345931 2020-09-12T14:14:02.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 85899345931 2020-09-12T14:14:02.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836495 1-42949672975 2-64424509452 3-85899345931' 2020-09-12T14:14:02.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:14:02.467 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2020-09-12T14:14:02.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=107374182409 2020-09-12T14:14:02.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 107374182409 2020-09-12T14:14:02.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836495 1-42949672975 2-64424509452 3-85899345931 4-107374182409' 2020-09-12T14:14:02.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:14:02.610 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836495 2020-09-12T14:14:02.611 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:14:02.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:14:02.611 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836495 2020-09-12T14:14:02.611 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:14:02.612 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836495 2020-09-12T14:14:02.612 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836495' 2020-09-12T14:14:02.612 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:14:02.612 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836495 2020-09-12T14:14:03.014 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836493 -lt 21474836495 2020-09-12T14:14:03.014 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:14:04.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:14:04.016 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:14:04.431 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836496 -lt 21474836495 2020-09-12T14:14:04.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:14:04.432 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672975 2020-09-12T14:14:04.432 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:14:04.433 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:14:04.433 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672975 2020-09-12T14:14:04.433 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:14:04.434 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672975 2020-09-12T14:14:04.434 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672975' 2020-09-12T14:14:04.434 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672975 2020-09-12T14:14:04.435 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:14:04.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672975 -lt 42949672975 2020-09-12T14:14:04.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:14:04.851 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509452 2020-09-12T14:14:04.851 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:14:04.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:14:04.852 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509452 2020-09-12T14:14:04.852 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:14:04.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509452 2020-09-12T14:14:04.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509452' 2020-09-12T14:14:04.853 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:14:04.853 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509452 2020-09-12T14:14:05.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509452 -lt 64424509452 2020-09-12T14:14:05.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:14:05.287 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 3-85899345931 2020-09-12T14:14:05.287 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:14:05.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=3 2020-09-12T14:14:05.288 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 3-85899345931 2020-09-12T14:14:05.288 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:14:05.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=85899345931 2020-09-12T14:14:05.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.3 seq 85899345931' 2020-09-12T14:14:05.289 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.3 seq 85899345931 2020-09-12T14:14:05.289 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 3 2020-09-12T14:14:05.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 85899345931 -lt 85899345931 2020-09-12T14:14:05.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:14:05.698 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 4-107374182409 2020-09-12T14:14:05.698 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:14:05.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=4 2020-09-12T14:14:05.700 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 4-107374182409 2020-09-12T14:14:05.700 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:14:05.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=107374182409 2020-09-12T14:14:05.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.4 seq 107374182409' 2020-09-12T14:14:05.701 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 4 2020-09-12T14:14:05.701 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.4 seq 107374182409 2020-09-12T14:14:06.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 107374182410 -lt 107374182409 2020-09-12T14:14:06.108 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:14:06.109 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:14:06.109 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:14:06.671 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 10 == 0 2020-09-12T14:14:06.672 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:14:06.672 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:14:06.672 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:14:06.672 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:14:06.672 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:14:06.672 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:14:06.673 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:14:07.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=10 2020-09-12T14:14:07.071 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:14:07.072 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:14:07.072 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:14:07.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 10 = 10 2020-09-12T14:14:07.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T14:14:07.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T14:14:07.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:71: TEST_backfill_priority: ceph pg dump pgs 2020-09-12T14:14:07.989 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:14:07.989 INFO:tasks.workunit.client.0.smithi027.stdout:10.0 0 0 0 0 0 0 0 0 0 0 active+clean 2020-09-12T14:13:57.411380+0000 0'0 81:19 [2,1] 2 [2,1] 2 0'0 2020-09-12T14:13:54.390272+0000 0'0 2020-09-12T14:13:54.390272+0000 0 2020-09-12T14:14:07.989 INFO:tasks.workunit.client.0.smithi027.stdout:9.0 0 0 0 0 0 0 0 0 0 0 active+clean 2020-09-12T14:13:54.794893+0000 0'0 80:25 [1,2] 1 [1,2] 1 0'0 2020-09-12T14:13:51.572991+0000 0'0 2020-09-12T14:13:51.572991+0000 0 2020-09-12T14:14:07.989 INFO:tasks.workunit.client.0.smithi027.stdout:8.0 0 0 0 0 0 0 0 0 0 0 active+clean 2020-09-12T14:13:52.186229+0000 0'0 80:25 [3,4] 3 [3,4] 3 0'0 2020-09-12T14:13:49.002623+0000 0'0 2020-09-12T14:13:49.002623+0000 0 2020-09-12T14:14:07.990 INFO:tasks.workunit.client.0.smithi027.stdout:7.0 0 0 0 0 0 0 0 0 0 0 active+clean 2020-09-12T14:13:49.100338+0000 0'0 80:32 [1,4] 1 [1,4] 1 0'0 2020-09-12T14:13:46.480650+0000 0'0 2020-09-12T14:13:46.480650+0000 0 2020-09-12T14:14:07.990 INFO:tasks.workunit.client.0.smithi027.stdout:2.0 0 0 0 0 0 0 0 0 0 0 active+clean 2020-09-12T14:13:36.544254+0000 0'0 80:50 [3,1] 3 [3,1] 3 0'0 2020-09-12T14:13:32.854534+0000 0'0 2020-09-12T14:13:32.854534+0000 0 2020-09-12T14:14:07.990 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 0 0 0 0 0 0 0 0 0 0 active+clean 2020-09-12T14:13:33.194564+0000 0'0 80:51 [1,0] 1 [1,0] 1 0'0 2020-09-12T14:13:30.255666+0000 0'0 2020-09-12T14:13:30.255666+0000 0 2020-09-12T14:14:07.990 INFO:tasks.workunit.client.0.smithi027.stdout:3.0 0 0 0 0 0 0 0 0 0 0 active+clean 2020-09-12T14:13:38.336634+0000 0'0 80:48 [1,2] 1 [1,2] 1 0'0 2020-09-12T14:13:35.363167+0000 0'0 2020-09-12T14:13:35.363167+0000 0 2020-09-12T14:14:07.990 INFO:tasks.workunit.client.0.smithi027.stdout:4.0 0 0 0 0 0 0 0 0 0 0 active+clean 2020-09-12T14:13:41.208204+0000 0'0 79:44 [4,0] 4 [4,0] 4 0'0 2020-09-12T14:13:38.529606+0000 0'0 2020-09-12T14:13:38.529606+0000 0 2020-09-12T14:14:07.990 INFO:tasks.workunit.client.0.smithi027.stdout:5.0 0 0 0 0 0 0 0 0 0 0 active+clean 2020-09-12T14:13:44.238656+0000 0'0 80:37 [3,4] 3 [3,4] 3 0'0 2020-09-12T14:13:41.230484+0000 0'0 2020-09-12T14:13:41.230484+0000 0 2020-09-12T14:14:07.990 INFO:tasks.workunit.client.0.smithi027.stdout:6.0 0 0 0 0 0 0 0 0 0 0 active+clean 2020-09-12T14:13:46.600755+0000 0'0 79:36 [4,2] 4 [4,2] 4 0'0 2020-09-12T14:13:43.794104+0000 0'0 2020-09-12T14:13:43.794104+0000 0 2020-09-12T14:14:07.991 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:14:07.991 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:14:07.991 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:14:08.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:75: TEST_backfill_priority: local PG1 2020-09-12T14:14:08.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:76: TEST_backfill_priority: local POOLNUM1 2020-09-12T14:14:08.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:77: TEST_backfill_priority: local pool1 2020-09-12T14:14:08.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:78: TEST_backfill_priority: local chk_osd1_1 2020-09-12T14:14:08.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:79: TEST_backfill_priority: local chk_osd1_2 2020-09-12T14:14:08.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:81: TEST_backfill_priority: local PG2 2020-09-12T14:14:08.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:82: TEST_backfill_priority: local POOLNUM2 2020-09-12T14:14:08.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:83: TEST_backfill_priority: local pool2 2020-09-12T14:14:08.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:84: TEST_backfill_priority: local chk_osd2 2020-09-12T14:14:08.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:86: TEST_backfill_priority: local PG3 2020-09-12T14:14:08.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:87: TEST_backfill_priority: local POOLNUM3 2020-09-12T14:14:08.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:88: TEST_backfill_priority: local pool3 2020-09-12T14:14:08.002 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:90: TEST_backfill_priority: seq 1 10 2020-09-12T14:14:08.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:90: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:14:08.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:92: TEST_backfill_priority: ceph pg map 1.0 --format=json 2020-09-12T14:14:08.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:92: TEST_backfill_priority: jq '.acting[]' 2020-09-12T14:14:08.402 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:93: TEST_backfill_priority: head -1 td/osd-backfill-prio/acting 2020-09-12T14:14:08.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:93: TEST_backfill_priority: local test_osd1=1 2020-09-12T14:14:08.403 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:94: TEST_backfill_priority: tail -1 td/osd-backfill-prio/acting 2020-09-12T14:14:08.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:94: TEST_backfill_priority: local test_osd2=0 2020-09-12T14:14:08.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:95: TEST_backfill_priority: '[' -z '' ']' 2020-09-12T14:14:08.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:97: TEST_backfill_priority: PG1=1.0 2020-09-12T14:14:08.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:98: TEST_backfill_priority: POOLNUM1=1 2020-09-12T14:14:08.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:99: TEST_backfill_priority: pool1=test1 2020-09-12T14:14:08.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:100: TEST_backfill_priority: chk_osd1_1=1 2020-09-12T14:14:08.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:101: TEST_backfill_priority: chk_osd1_2=0 2020-09-12T14:14:08.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:90: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:14:08.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:92: TEST_backfill_priority: ceph pg map 2.0 --format=json 2020-09-12T14:14:08.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:92: TEST_backfill_priority: jq '.acting[]' 2020-09-12T14:14:08.804 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:93: TEST_backfill_priority: head -1 td/osd-backfill-prio/acting 2020-09-12T14:14:08.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:93: TEST_backfill_priority: local test_osd1=3 2020-09-12T14:14:08.806 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:94: TEST_backfill_priority: tail -1 td/osd-backfill-prio/acting 2020-09-12T14:14:08.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:94: TEST_backfill_priority: local test_osd2=1 2020-09-12T14:14:08.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:95: TEST_backfill_priority: '[' -z 1.0 ']' 2020-09-12T14:14:08.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:102: TEST_backfill_priority: '[' -z '' -a 1 = 3 -a 0 '!=' 1 ']' 2020-09-12T14:14:08.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:108: TEST_backfill_priority: '[' -n '' -a 1 = 3 -a 0 '!=' 1 -a '' '!=' 1 ']' 2020-09-12T14:14:08.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:90: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:14:08.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:92: TEST_backfill_priority: ceph pg map 3.0 --format=json 2020-09-12T14:14:08.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:92: TEST_backfill_priority: jq '.acting[]' 2020-09-12T14:14:09.203 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:93: TEST_backfill_priority: head -1 td/osd-backfill-prio/acting 2020-09-12T14:14:09.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:93: TEST_backfill_priority: local test_osd1=1 2020-09-12T14:14:09.203 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:94: TEST_backfill_priority: tail -1 td/osd-backfill-prio/acting 2020-09-12T14:14:09.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:94: TEST_backfill_priority: local test_osd2=2 2020-09-12T14:14:09.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:95: TEST_backfill_priority: '[' -z 1.0 ']' 2020-09-12T14:14:09.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:102: TEST_backfill_priority: '[' -z '' -a 1 = 1 -a 0 '!=' 2 ']' 2020-09-12T14:14:09.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:104: TEST_backfill_priority: PG2=3.0 2020-09-12T14:14:09.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:105: TEST_backfill_priority: POOLNUM2=3 2020-09-12T14:14:09.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:106: TEST_backfill_priority: pool2=test3 2020-09-12T14:14:09.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:107: TEST_backfill_priority: chk_osd2=2 2020-09-12T14:14:09.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:90: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:14:09.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:92: TEST_backfill_priority: ceph pg map 4.0 --format=json 2020-09-12T14:14:09.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:92: TEST_backfill_priority: jq '.acting[]' 2020-09-12T14:14:09.636 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:93: TEST_backfill_priority: head -1 td/osd-backfill-prio/acting 2020-09-12T14:14:09.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:93: TEST_backfill_priority: local test_osd1=4 2020-09-12T14:14:09.637 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:94: TEST_backfill_priority: tail -1 td/osd-backfill-prio/acting 2020-09-12T14:14:09.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:94: TEST_backfill_priority: local test_osd2=0 2020-09-12T14:14:09.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:95: TEST_backfill_priority: '[' -z 1.0 ']' 2020-09-12T14:14:09.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:102: TEST_backfill_priority: '[' -z 3.0 -a 1 = 4 -a 0 '!=' 0 ']' 2020-09-12T14:14:09.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:108: TEST_backfill_priority: '[' -n 3.0 -a 1 = 4 -a 0 '!=' 0 -a 2 '!=' 0 ']' 2020-09-12T14:14:09.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:90: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:14:09.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:92: TEST_backfill_priority: ceph pg map 5.0 --format=json 2020-09-12T14:14:09.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:92: TEST_backfill_priority: jq '.acting[]' 2020-09-12T14:14:10.049 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:93: TEST_backfill_priority: head -1 td/osd-backfill-prio/acting 2020-09-12T14:14:10.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:93: TEST_backfill_priority: local test_osd1=3 2020-09-12T14:14:10.050 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:94: TEST_backfill_priority: tail -1 td/osd-backfill-prio/acting 2020-09-12T14:14:10.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:94: TEST_backfill_priority: local test_osd2=4 2020-09-12T14:14:10.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:95: TEST_backfill_priority: '[' -z 1.0 ']' 2020-09-12T14:14:10.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:102: TEST_backfill_priority: '[' -z 3.0 -a 1 = 3 -a 0 '!=' 4 ']' 2020-09-12T14:14:10.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:108: TEST_backfill_priority: '[' -n 3.0 -a 1 = 3 -a 0 '!=' 4 -a 2 '!=' 4 ']' 2020-09-12T14:14:10.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:90: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:14:10.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:92: TEST_backfill_priority: ceph pg map 6.0 --format=json 2020-09-12T14:14:10.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:92: TEST_backfill_priority: jq '.acting[]' 2020-09-12T14:14:10.464 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:93: TEST_backfill_priority: head -1 td/osd-backfill-prio/acting 2020-09-12T14:14:10.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:93: TEST_backfill_priority: local test_osd1=4 2020-09-12T14:14:10.465 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:94: TEST_backfill_priority: tail -1 td/osd-backfill-prio/acting 2020-09-12T14:14:10.466 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:94: TEST_backfill_priority: local test_osd2=2 2020-09-12T14:14:10.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:95: TEST_backfill_priority: '[' -z 1.0 ']' 2020-09-12T14:14:10.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:102: TEST_backfill_priority: '[' -z 3.0 -a 1 = 4 -a 0 '!=' 2 ']' 2020-09-12T14:14:10.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:108: TEST_backfill_priority: '[' -n 3.0 -a 1 = 4 -a 0 '!=' 2 -a 2 '!=' 2 ']' 2020-09-12T14:14:10.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:90: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:14:10.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:92: TEST_backfill_priority: ceph pg map 7.0 --format=json 2020-09-12T14:14:10.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:92: TEST_backfill_priority: jq '.acting[]' 2020-09-12T14:14:10.881 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:93: TEST_backfill_priority: head -1 td/osd-backfill-prio/acting 2020-09-12T14:14:10.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:93: TEST_backfill_priority: local test_osd1=1 2020-09-12T14:14:10.881 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:94: TEST_backfill_priority: tail -1 td/osd-backfill-prio/acting 2020-09-12T14:14:10.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:94: TEST_backfill_priority: local test_osd2=4 2020-09-12T14:14:10.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:95: TEST_backfill_priority: '[' -z 1.0 ']' 2020-09-12T14:14:10.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:102: TEST_backfill_priority: '[' -z 3.0 -a 1 = 1 -a 0 '!=' 4 ']' 2020-09-12T14:14:10.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:108: TEST_backfill_priority: '[' -n 3.0 -a 1 = 1 -a 0 '!=' 4 -a 2 '!=' 4 ']' 2020-09-12T14:14:10.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:110: TEST_backfill_priority: PG3=7.0 2020-09-12T14:14:10.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:111: TEST_backfill_priority: POOLNUM3=7 2020-09-12T14:14:10.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:112: TEST_backfill_priority: pool3=test7 2020-09-12T14:14:10.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:113: TEST_backfill_priority: break 2020-09-12T14:14:10.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:116: TEST_backfill_priority: rm -f td/osd-backfill-prio/acting 2020-09-12T14:14:10.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:118: TEST_backfill_priority: '[' test3 = '' -o pool3 = '' ']' 2020-09-12T14:14:10.884 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:124: TEST_backfill_priority: seq 1 10 2020-09-12T14:14:10.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:124: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:14:10.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:126: TEST_backfill_priority: '[' 1 '!=' 1 -a 1 '!=' 3 -a 1 '!=' 7 ']' 2020-09-12T14:14:10.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:124: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:14:10.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:126: TEST_backfill_priority: '[' 2 '!=' 1 -a 2 '!=' 3 -a 2 '!=' 7 ']' 2020-09-12T14:14:10.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:128: TEST_backfill_priority: delete_pool test2 2020-09-12T14:14:10.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: delete_pool: local poolname=test2 2020-09-12T14:14:10.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: ceph osd pool delete test2 test2 --yes-i-really-really-mean-it 2020-09-12T14:14:11.454 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test2' does not exist 2020-09-12T14:14:11.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:124: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:14:11.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:126: TEST_backfill_priority: '[' 3 '!=' 1 -a 3 '!=' 3 -a 3 '!=' 7 ']' 2020-09-12T14:14:11.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:124: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:14:11.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:126: TEST_backfill_priority: '[' 4 '!=' 1 -a 4 '!=' 3 -a 4 '!=' 7 ']' 2020-09-12T14:14:11.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:128: TEST_backfill_priority: delete_pool test4 2020-09-12T14:14:11.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: delete_pool: local poolname=test4 2020-09-12T14:14:11.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: ceph osd pool delete test4 test4 --yes-i-really-really-mean-it 2020-09-12T14:14:12.122 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test4' does not exist 2020-09-12T14:14:12.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:124: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:14:12.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:126: TEST_backfill_priority: '[' 5 '!=' 1 -a 5 '!=' 3 -a 5 '!=' 7 ']' 2020-09-12T14:14:12.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:128: TEST_backfill_priority: delete_pool test5 2020-09-12T14:14:12.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: delete_pool: local poolname=test5 2020-09-12T14:14:12.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: ceph osd pool delete test5 test5 --yes-i-really-really-mean-it 2020-09-12T14:14:12.673 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test5' does not exist 2020-09-12T14:14:12.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:124: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:14:12.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:126: TEST_backfill_priority: '[' 6 '!=' 1 -a 6 '!=' 3 -a 6 '!=' 7 ']' 2020-09-12T14:14:12.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:128: TEST_backfill_priority: delete_pool test6 2020-09-12T14:14:12.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: delete_pool: local poolname=test6 2020-09-12T14:14:12.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: ceph osd pool delete test6 test6 --yes-i-really-really-mean-it 2020-09-12T14:14:13.303 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test6' does not exist 2020-09-12T14:14:13.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:124: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:14:13.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:126: TEST_backfill_priority: '[' 7 '!=' 1 -a 7 '!=' 3 -a 7 '!=' 7 ']' 2020-09-12T14:14:13.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:124: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:14:13.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:126: TEST_backfill_priority: '[' 8 '!=' 1 -a 8 '!=' 3 -a 8 '!=' 7 ']' 2020-09-12T14:14:13.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:128: TEST_backfill_priority: delete_pool test8 2020-09-12T14:14:13.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: delete_pool: local poolname=test8 2020-09-12T14:14:13.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: ceph osd pool delete test8 test8 --yes-i-really-really-mean-it 2020-09-12T14:14:14.006 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test8' does not exist 2020-09-12T14:14:14.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:124: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:14:14.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:126: TEST_backfill_priority: '[' 9 '!=' 1 -a 9 '!=' 3 -a 9 '!=' 7 ']' 2020-09-12T14:14:14.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:128: TEST_backfill_priority: delete_pool test9 2020-09-12T14:14:14.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: delete_pool: local poolname=test9 2020-09-12T14:14:14.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: ceph osd pool delete test9 test9 --yes-i-really-really-mean-it 2020-09-12T14:14:14.676 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test9' does not exist 2020-09-12T14:14:14.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:124: TEST_backfill_priority: for p in $(seq 1 $pools) 2020-09-12T14:14:14.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:126: TEST_backfill_priority: '[' 10 '!=' 1 -a 10 '!=' 3 -a 10 '!=' 7 ']' 2020-09-12T14:14:14.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:128: TEST_backfill_priority: delete_pool test10 2020-09-12T14:14:14.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: delete_pool: local poolname=test10 2020-09-12T14:14:14.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: ceph osd pool delete test10 test10 --yes-i-really-really-mean-it 2020-09-12T14:14:15.437 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test10' does not exist 2020-09-12T14:14:15.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:132: TEST_backfill_priority: ceph osd pool set test3 size 1 --yes-i-really-mean-it 2020-09-12T14:14:16.226 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 3 size to 1 2020-09-12T14:14:16.236 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:133: TEST_backfill_priority: ceph osd pool set test7 size 1 --yes-i-really-mean-it 2020-09-12T14:14:17.012 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 7 size to 1 2020-09-12T14:14:17.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:134: TEST_backfill_priority: wait_for_clean 2020-09-12T14:14:17.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:14:17.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:14:17.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:14:17.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:14:17.022 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 90 .1 2020-09-12T14:14:17.023 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:14:17.023 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:14:17.023 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:14:17.024 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:14:17.024 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:14:17.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:14:17.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:14:17.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:14:17.119 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:14:17.119 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:14:17.590 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:14:17.591 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:14:17.591 INFO:tasks.workunit.client.0.smithi027.stderr:2 2020-09-12T14:14:17.591 INFO:tasks.workunit.client.0.smithi027.stderr:3 2020-09-12T14:14:17.592 INFO:tasks.workunit.client.0.smithi027.stderr:4' 2020-09-12T14:14:17.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:14:17.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:14:17.593 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:14:17.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836500 2020-09-12T14:14:17.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836500 2020-09-12T14:14:17.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836500' 2020-09-12T14:14:17.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:14:17.735 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:14:17.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672980 2020-09-12T14:14:17.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672980 2020-09-12T14:14:17.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836500 1-42949672980' 2020-09-12T14:14:17.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:14:17.879 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:14:18.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509457 2020-09-12T14:14:18.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509457 2020-09-12T14:14:18.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836500 1-42949672980 2-64424509457' 2020-09-12T14:14:18.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:14:18.023 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2020-09-12T14:14:18.165 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=85899345936 2020-09-12T14:14:18.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 85899345936 2020-09-12T14:14:18.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836500 1-42949672980 2-64424509457 3-85899345936' 2020-09-12T14:14:18.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:14:18.166 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2020-09-12T14:14:18.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=107374182414 2020-09-12T14:14:18.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 107374182414 2020-09-12T14:14:18.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836500 1-42949672980 2-64424509457 3-85899345936 4-107374182414' 2020-09-12T14:14:18.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:14:18.310 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836500 2020-09-12T14:14:18.310 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:14:18.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:14:18.312 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836500 2020-09-12T14:14:18.312 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:14:18.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836500 2020-09-12T14:14:18.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836500' 2020-09-12T14:14:18.313 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:14:18.313 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836500 2020-09-12T14:14:18.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836498 -lt 21474836500 2020-09-12T14:14:18.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:14:19.728 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:14:19.728 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:14:20.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836501 -lt 21474836500 2020-09-12T14:14:20.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:14:20.145 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672980 2020-09-12T14:14:20.145 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:14:20.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:14:20.146 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672980 2020-09-12T14:14:20.146 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:14:20.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672980 2020-09-12T14:14:20.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672980' 2020-09-12T14:14:20.147 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672980 2020-09-12T14:14:20.147 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:14:20.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672980 -lt 42949672980 2020-09-12T14:14:20.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:14:20.570 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509457 2020-09-12T14:14:20.570 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:14:20.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:14:20.571 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509457 2020-09-12T14:14:20.571 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:14:20.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509457 2020-09-12T14:14:20.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509457' 2020-09-12T14:14:20.572 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509457 2020-09-12T14:14:20.573 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:14:20.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509458 -lt 64424509457 2020-09-12T14:14:20.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:14:20.993 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 3-85899345936 2020-09-12T14:14:20.993 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:14:20.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=3 2020-09-12T14:14:20.994 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 3-85899345936 2020-09-12T14:14:20.994 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:14:20.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=85899345936 2020-09-12T14:14:20.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.3 seq 85899345936' 2020-09-12T14:14:20.995 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.3 seq 85899345936 2020-09-12T14:14:20.995 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 3 2020-09-12T14:14:21.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 85899345936 -lt 85899345936 2020-09-12T14:14:21.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:14:21.414 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 4-107374182414 2020-09-12T14:14:21.414 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:14:21.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=4 2020-09-12T14:14:21.415 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 4-107374182414 2020-09-12T14:14:21.415 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:14:21.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=107374182414 2020-09-12T14:14:21.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.4 seq 107374182414' 2020-09-12T14:14:21.416 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.4 seq 107374182414 2020-09-12T14:14:21.417 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 4 2020-09-12T14:14:21.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 107374182415 -lt 107374182414 2020-09-12T14:14:21.834 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:14:21.834 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:14:21.834 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:14:22.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 3 == 0 2020-09-12T14:14:22.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:14:22.361 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:14:22.361 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:14:22.362 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:14:22.362 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:14:22.362 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:14:22.362 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:14:22.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=3 2020-09-12T14:14:22.758 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:14:22.759 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:14:22.759 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:14:23.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 3 = 3 2020-09-12T14:14:23.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T14:14:23.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T14:14:23.291 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:136: TEST_backfill_priority: dd if=/dev/urandom of=td/osd-backfill-prio/data bs=1M count=10 2020-09-12T14:14:23.338 INFO:tasks.workunit.client.0.smithi027.stderr:10+0 records in 2020-09-12T14:14:23.339 INFO:tasks.workunit.client.0.smithi027.stderr:10+0 records out 2020-09-12T14:14:23.339 INFO:tasks.workunit.client.0.smithi027.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0469091 s, 224 MB/s 2020-09-12T14:14:23.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:137: TEST_backfill_priority: p=1 2020-09-12T14:14:23.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:138: TEST_backfill_priority: for pname in $pool1 $pool2 $pool3 2020-09-12T14:14:23.340 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: seq 1 50 2020-09-12T14:14:23.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:23.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj1-p1 td/osd-backfill-prio/data 2020-09-12T14:14:24.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:24.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj2-p1 td/osd-backfill-prio/data 2020-09-12T14:14:24.762 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:24.762 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj3-p1 td/osd-backfill-prio/data 2020-09-12T14:14:25.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:25.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj4-p1 td/osd-backfill-prio/data 2020-09-12T14:14:26.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:26.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj5-p1 td/osd-backfill-prio/data 2020-09-12T14:14:26.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:26.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj6-p1 td/osd-backfill-prio/data 2020-09-12T14:14:27.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:27.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj7-p1 td/osd-backfill-prio/data 2020-09-12T14:14:28.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:28.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj8-p1 td/osd-backfill-prio/data 2020-09-12T14:14:29.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:29.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj9-p1 td/osd-backfill-prio/data 2020-09-12T14:14:30.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:30.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj10-p1 td/osd-backfill-prio/data 2020-09-12T14:14:30.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:30.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj11-p1 td/osd-backfill-prio/data 2020-09-12T14:14:31.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:31.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj12-p1 td/osd-backfill-prio/data 2020-09-12T14:14:32.315 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:32.315 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj13-p1 td/osd-backfill-prio/data 2020-09-12T14:14:33.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:33.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj14-p1 td/osd-backfill-prio/data 2020-09-12T14:14:33.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:33.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj15-p1 td/osd-backfill-prio/data 2020-09-12T14:14:34.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:34.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj16-p1 td/osd-backfill-prio/data 2020-09-12T14:14:35.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:35.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj17-p1 td/osd-backfill-prio/data 2020-09-12T14:14:36.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:36.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj18-p1 td/osd-backfill-prio/data 2020-09-12T14:14:36.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:36.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj19-p1 td/osd-backfill-prio/data 2020-09-12T14:14:37.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:37.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj20-p1 td/osd-backfill-prio/data 2020-09-12T14:14:38.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:38.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj21-p1 td/osd-backfill-prio/data 2020-09-12T14:14:39.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:39.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj22-p1 td/osd-backfill-prio/data 2020-09-12T14:14:39.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:39.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj23-p1 td/osd-backfill-prio/data 2020-09-12T14:14:40.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:40.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj24-p1 td/osd-backfill-prio/data 2020-09-12T14:14:42.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:42.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj25-p1 td/osd-backfill-prio/data 2020-09-12T14:14:43.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:43.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj26-p1 td/osd-backfill-prio/data 2020-09-12T14:14:43.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:43.671 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj27-p1 td/osd-backfill-prio/data 2020-09-12T14:14:44.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:44.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj28-p1 td/osd-backfill-prio/data 2020-09-12T14:14:45.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:45.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj29-p1 td/osd-backfill-prio/data 2020-09-12T14:14:46.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:46.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj30-p1 td/osd-backfill-prio/data 2020-09-12T14:14:47.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:47.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj31-p1 td/osd-backfill-prio/data 2020-09-12T14:14:47.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:47.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj32-p1 td/osd-backfill-prio/data 2020-09-12T14:14:48.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:48.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj33-p1 td/osd-backfill-prio/data 2020-09-12T14:14:49.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:49.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj34-p1 td/osd-backfill-prio/data 2020-09-12T14:14:50.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:50.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj35-p1 td/osd-backfill-prio/data 2020-09-12T14:14:50.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:50.765 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj36-p1 td/osd-backfill-prio/data 2020-09-12T14:14:51.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:51.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj37-p1 td/osd-backfill-prio/data 2020-09-12T14:14:52.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:52.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj38-p1 td/osd-backfill-prio/data 2020-09-12T14:14:53.196 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:53.196 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj39-p1 td/osd-backfill-prio/data 2020-09-12T14:14:53.823 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:53.823 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj40-p1 td/osd-backfill-prio/data 2020-09-12T14:14:55.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:55.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj41-p1 td/osd-backfill-prio/data 2020-09-12T14:14:55.661 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:55.661 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj42-p1 td/osd-backfill-prio/data 2020-09-12T14:14:56.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:56.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj43-p1 td/osd-backfill-prio/data 2020-09-12T14:14:57.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:57.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj44-p1 td/osd-backfill-prio/data 2020-09-12T14:14:58.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:58.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj45-p1 td/osd-backfill-prio/data 2020-09-12T14:14:58.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:58.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj46-p1 td/osd-backfill-prio/data 2020-09-12T14:14:59.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:14:59.547 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj47-p1 td/osd-backfill-prio/data 2020-09-12T14:15:01.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:01.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj48-p1 td/osd-backfill-prio/data 2020-09-12T14:15:01.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:01.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj49-p1 td/osd-backfill-prio/data 2020-09-12T14:15:02.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:02.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test1 put obj50-p1 td/osd-backfill-prio/data 2020-09-12T14:15:03.190 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:144: TEST_backfill_priority: expr 1 + 1 2020-09-12T14:15:03.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:144: TEST_backfill_priority: p=2 2020-09-12T14:15:03.191 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:138: TEST_backfill_priority: for pname in $pool1 $pool2 $pool3 2020-09-12T14:15:03.191 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: seq 1 50 2020-09-12T14:15:03.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:03.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj1-p2 td/osd-backfill-prio/data 2020-09-12T14:15:03.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:03.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj2-p2 td/osd-backfill-prio/data 2020-09-12T14:15:04.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:04.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj3-p2 td/osd-backfill-prio/data 2020-09-12T14:15:04.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:04.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj4-p2 td/osd-backfill-prio/data 2020-09-12T14:15:05.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:05.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj5-p2 td/osd-backfill-prio/data 2020-09-12T14:15:05.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:05.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj6-p2 td/osd-backfill-prio/data 2020-09-12T14:15:06.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:06.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj7-p2 td/osd-backfill-prio/data 2020-09-12T14:15:06.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:06.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj8-p2 td/osd-backfill-prio/data 2020-09-12T14:15:06.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:06.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj9-p2 td/osd-backfill-prio/data 2020-09-12T14:15:07.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:07.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj10-p2 td/osd-backfill-prio/data 2020-09-12T14:15:07.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:07.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj11-p2 td/osd-backfill-prio/data 2020-09-12T14:15:08.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:08.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj12-p2 td/osd-backfill-prio/data 2020-09-12T14:15:08.762 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:08.762 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj13-p2 td/osd-backfill-prio/data 2020-09-12T14:15:09.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:09.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj14-p2 td/osd-backfill-prio/data 2020-09-12T14:15:09.422 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:09.423 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj15-p2 td/osd-backfill-prio/data 2020-09-12T14:15:09.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:09.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj16-p2 td/osd-backfill-prio/data 2020-09-12T14:15:10.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:10.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj17-p2 td/osd-backfill-prio/data 2020-09-12T14:15:10.550 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:10.551 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj18-p2 td/osd-backfill-prio/data 2020-09-12T14:15:10.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:10.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj19-p2 td/osd-backfill-prio/data 2020-09-12T14:15:11.848 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:11.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj20-p2 td/osd-backfill-prio/data 2020-09-12T14:15:12.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:12.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj21-p2 td/osd-backfill-prio/data 2020-09-12T14:15:12.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:12.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj22-p2 td/osd-backfill-prio/data 2020-09-12T14:15:13.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:13.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj23-p2 td/osd-backfill-prio/data 2020-09-12T14:15:13.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:13.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj24-p2 td/osd-backfill-prio/data 2020-09-12T14:15:13.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:13.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj25-p2 td/osd-backfill-prio/data 2020-09-12T14:15:14.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:14.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj26-p2 td/osd-backfill-prio/data 2020-09-12T14:15:14.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:14.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj27-p2 td/osd-backfill-prio/data 2020-09-12T14:15:15.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:15.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj28-p2 td/osd-backfill-prio/data 2020-09-12T14:15:15.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:15.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj29-p2 td/osd-backfill-prio/data 2020-09-12T14:15:16.090 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:16.090 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj30-p2 td/osd-backfill-prio/data 2020-09-12T14:15:16.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:16.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj31-p2 td/osd-backfill-prio/data 2020-09-12T14:15:16.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:16.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj32-p2 td/osd-backfill-prio/data 2020-09-12T14:15:17.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:17.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj33-p2 td/osd-backfill-prio/data 2020-09-12T14:15:17.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:17.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj34-p2 td/osd-backfill-prio/data 2020-09-12T14:15:18.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:18.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj35-p2 td/osd-backfill-prio/data 2020-09-12T14:15:18.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:18.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj36-p2 td/osd-backfill-prio/data 2020-09-12T14:15:18.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:18.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj37-p2 td/osd-backfill-prio/data 2020-09-12T14:15:19.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:19.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj38-p2 td/osd-backfill-prio/data 2020-09-12T14:15:19.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:19.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj39-p2 td/osd-backfill-prio/data 2020-09-12T14:15:20.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:20.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj40-p2 td/osd-backfill-prio/data 2020-09-12T14:15:20.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:20.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj41-p2 td/osd-backfill-prio/data 2020-09-12T14:15:21.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:21.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj42-p2 td/osd-backfill-prio/data 2020-09-12T14:15:21.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:21.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj43-p2 td/osd-backfill-prio/data 2020-09-12T14:15:22.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:22.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj44-p2 td/osd-backfill-prio/data 2020-09-12T14:15:22.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:22.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj45-p2 td/osd-backfill-prio/data 2020-09-12T14:15:23.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:23.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj46-p2 td/osd-backfill-prio/data 2020-09-12T14:15:24.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:24.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj47-p2 td/osd-backfill-prio/data 2020-09-12T14:15:26.079 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:26.079 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj48-p2 td/osd-backfill-prio/data 2020-09-12T14:15:26.438 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:26.439 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj49-p2 td/osd-backfill-prio/data 2020-09-12T14:15:26.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:26.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test3 put obj50-p2 td/osd-backfill-prio/data 2020-09-12T14:15:27.520 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:144: TEST_backfill_priority: expr 2 + 1 2020-09-12T14:15:27.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:144: TEST_backfill_priority: p=3 2020-09-12T14:15:27.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:138: TEST_backfill_priority: for pname in $pool1 $pool2 $pool3 2020-09-12T14:15:27.521 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: seq 1 50 2020-09-12T14:15:27.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:27.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj1-p3 td/osd-backfill-prio/data 2020-09-12T14:15:27.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:27.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj2-p3 td/osd-backfill-prio/data 2020-09-12T14:15:28.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:28.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj3-p3 td/osd-backfill-prio/data 2020-09-12T14:15:28.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:28.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj4-p3 td/osd-backfill-prio/data 2020-09-12T14:15:29.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:29.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj5-p3 td/osd-backfill-prio/data 2020-09-12T14:15:29.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:29.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj6-p3 td/osd-backfill-prio/data 2020-09-12T14:15:30.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:30.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj7-p3 td/osd-backfill-prio/data 2020-09-12T14:15:30.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:30.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj8-p3 td/osd-backfill-prio/data 2020-09-12T14:15:31.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:31.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj9-p3 td/osd-backfill-prio/data 2020-09-12T14:15:31.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:31.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj10-p3 td/osd-backfill-prio/data 2020-09-12T14:15:31.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:31.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj11-p3 td/osd-backfill-prio/data 2020-09-12T14:15:32.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:32.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj12-p3 td/osd-backfill-prio/data 2020-09-12T14:15:32.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:32.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj13-p3 td/osd-backfill-prio/data 2020-09-12T14:15:33.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:33.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj14-p3 td/osd-backfill-prio/data 2020-09-12T14:15:33.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:33.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj15-p3 td/osd-backfill-prio/data 2020-09-12T14:15:34.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:34.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj16-p3 td/osd-backfill-prio/data 2020-09-12T14:15:34.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:34.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj17-p3 td/osd-backfill-prio/data 2020-09-12T14:15:34.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:34.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj18-p3 td/osd-backfill-prio/data 2020-09-12T14:15:36.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:36.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj19-p3 td/osd-backfill-prio/data 2020-09-12T14:15:36.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:36.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj20-p3 td/osd-backfill-prio/data 2020-09-12T14:15:37.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:37.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj21-p3 td/osd-backfill-prio/data 2020-09-12T14:15:37.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:37.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj22-p3 td/osd-backfill-prio/data 2020-09-12T14:15:37.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:37.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj23-p3 td/osd-backfill-prio/data 2020-09-12T14:15:38.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:38.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj24-p3 td/osd-backfill-prio/data 2020-09-12T14:15:38.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:38.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj25-p3 td/osd-backfill-prio/data 2020-09-12T14:15:38.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:38.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj26-p3 td/osd-backfill-prio/data 2020-09-12T14:15:39.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:39.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj27-p3 td/osd-backfill-prio/data 2020-09-12T14:15:40.262 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:40.262 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj28-p3 td/osd-backfill-prio/data 2020-09-12T14:15:40.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:40.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj29-p3 td/osd-backfill-prio/data 2020-09-12T14:15:40.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:40.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj30-p3 td/osd-backfill-prio/data 2020-09-12T14:15:41.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:41.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj31-p3 td/osd-backfill-prio/data 2020-09-12T14:15:41.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:41.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj32-p3 td/osd-backfill-prio/data 2020-09-12T14:15:42.093 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:42.093 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj33-p3 td/osd-backfill-prio/data 2020-09-12T14:15:42.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:42.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj34-p3 td/osd-backfill-prio/data 2020-09-12T14:15:43.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:43.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj35-p3 td/osd-backfill-prio/data 2020-09-12T14:15:43.622 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:43.622 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj36-p3 td/osd-backfill-prio/data 2020-09-12T14:15:44.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:44.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj37-p3 td/osd-backfill-prio/data 2020-09-12T14:15:44.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:44.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj38-p3 td/osd-backfill-prio/data 2020-09-12T14:15:44.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:44.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj39-p3 td/osd-backfill-prio/data 2020-09-12T14:15:45.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:45.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj40-p3 td/osd-backfill-prio/data 2020-09-12T14:15:46.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:46.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj41-p3 td/osd-backfill-prio/data 2020-09-12T14:15:46.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:46.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj42-p3 td/osd-backfill-prio/data 2020-09-12T14:15:47.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:47.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj43-p3 td/osd-backfill-prio/data 2020-09-12T14:15:47.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:47.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj44-p3 td/osd-backfill-prio/data 2020-09-12T14:15:47.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:47.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj45-p3 td/osd-backfill-prio/data 2020-09-12T14:15:48.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:48.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj46-p3 td/osd-backfill-prio/data 2020-09-12T14:15:48.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:48.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj47-p3 td/osd-backfill-prio/data 2020-09-12T14:15:49.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:49.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj48-p3 td/osd-backfill-prio/data 2020-09-12T14:15:49.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:49.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj49-p3 td/osd-backfill-prio/data 2020-09-12T14:15:50.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:140: TEST_backfill_priority: for i in $(seq 1 $objects) 2020-09-12T14:15:50.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:142: TEST_backfill_priority: rados -p test7 put obj50-p3 td/osd-backfill-prio/data 2020-09-12T14:15:50.332 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:144: TEST_backfill_priority: expr 3 + 1 2020-09-12T14:15:50.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:144: TEST_backfill_priority: p=4 2020-09-12T14:15:50.333 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:147: TEST_backfill_priority: get_not_primary test1 obj1-p1 2020-09-12T14:15:50.333 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1167: get_not_primary: local poolname=test1 2020-09-12T14:15:50.333 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: get_not_primary: local objectname=obj1-p1 2020-09-12T14:15:50.334 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1170: get_not_primary: get_primary test1 obj1-p1 2020-09-12T14:15:50.334 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1134: get_primary: local poolname=test1 2020-09-12T14:15:50.334 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1135: get_primary: local objectname=obj1-p1 2020-09-12T14:15:50.334 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1137: get_primary: ceph --format json osd map test1 obj1-p1 2020-09-12T14:15:50.335 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1138: get_primary: jq .acting_primary 2020-09-12T14:15:50.754 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1170: get_not_primary: local primary=1 2020-09-12T14:15:50.754 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1171: get_not_primary: ceph --format json osd map test1 obj1-p1 2020-09-12T14:15:50.755 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1172: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2020-09-12T14:15:51.164 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:147: TEST_backfill_priority: local otherosd=0 2020-09-12T14:15:51.165 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:149: TEST_backfill_priority: ceph pg dump pgs 2020-09-12T14:15:51.748 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:15:51.749 INFO:tasks.workunit.client.0.smithi027.stdout:7.0 50 0 0 0 0 524288000 0 0 50 50 active+clean 2020-09-12T14:14:17.696345+0000 93'150 93:195 [1] 1 [1] 1 0'0 2020-09-12T14:13:46.480650+0000 0'0 2020-09-12T14:13:46.480650+0000 0 2020-09-12T14:15:51.749 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 50 0 0 0 0 524288000 0 0 50 50 active+clean 2020-09-12T14:13:33.194564+0000 93'150 93:207 [1,0] 1 [1,0] 1 0'0 2020-09-12T14:13:30.255666+0000 0'0 2020-09-12T14:13:30.255666+0000 0 2020-09-12T14:15:51.749 INFO:tasks.workunit.client.0.smithi027.stdout:3.0 50 0 0 0 0 524288000 0 0 50 50 active+clean 2020-09-12T14:14:17.841130+0000 93'150 93:211 [1] 1 [1] 1 0'0 2020-09-12T14:13:35.363167+0000 0'0 2020-09-12T14:13:35.363167+0000 0 2020-09-12T14:15:51.749 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:15:51.749 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:15:51.750 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:15:51.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:150: TEST_backfill_priority: ERRORS=0 2020-09-12T14:15:51.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:152: TEST_backfill_priority: ceph osd set nobackfill 2020-09-12T14:15:52.494 INFO:tasks.workunit.client.0.smithi027.stderr:nobackfill is set 2020-09-12T14:15:52.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:153: TEST_backfill_priority: ceph osd set noout 2020-09-12T14:15:53.415 INFO:tasks.workunit.client.0.smithi027.stderr:noout is set 2020-09-12T14:15:53.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:157: TEST_backfill_priority: ceph osd pool set test7 size 2 2020-09-12T14:15:54.183 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 7 size to 2 2020-09-12T14:15:54.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:158: TEST_backfill_priority: sleep 2 2020-09-12T14:15:56.195 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:160: TEST_backfill_priority: get_asok_path osd.1 2020-09-12T14:15:56.195 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2020-09-12T14:15:56.196 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2020-09-12T14:15:56.196 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:15:56.196 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:15:56.196 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:15:56.197 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.73258/ceph-osd.1.asok 2020-09-12T14:15:56.197 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:160: TEST_backfill_priority: CEPH_ARGS= 2020-09-12T14:15:56.197 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:160: TEST_backfill_priority: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:15:56.310 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:15:56.311 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:15:56.311 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:15:56.311 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:15:56.311 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:15:56.311 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:15:56.311 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:15:56.312 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:15:56.312 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:15:56.312 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:15:56.312 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:15:56.312 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:15:56.312 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:15:56.312 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:15:56.320 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:163: TEST_backfill_priority: seq 1 10 2020-09-12T14:15:56.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:163: TEST_backfill_priority: for i in $(seq 1 $max_tries) 2020-09-12T14:15:56.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:165: TEST_backfill_priority: ceph pg force-backfill 7.0 2020-09-12T14:15:56.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:165: TEST_backfill_priority: grep -q 'doesn'\''t require backfilling' 2020-09-12T14:15:56.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:168: TEST_backfill_priority: '[' 1 = 10 ']' 2020-09-12T14:15:56.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:172: TEST_backfill_priority: sleep 2 2020-09-12T14:15:58.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:163: TEST_backfill_priority: for i in $(seq 1 $max_tries) 2020-09-12T14:15:58.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:165: TEST_backfill_priority: ceph pg force-backfill 7.0 2020-09-12T14:15:58.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:165: TEST_backfill_priority: grep -q 'doesn'\''t require backfilling' 2020-09-12T14:15:59.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:168: TEST_backfill_priority: '[' 2 = 10 ']' 2020-09-12T14:15:59.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:172: TEST_backfill_priority: sleep 2 2020-09-12T14:16:01.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:163: TEST_backfill_priority: for i in $(seq 1 $max_tries) 2020-09-12T14:16:01.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:165: TEST_backfill_priority: ceph pg force-backfill 7.0 2020-09-12T14:16:01.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:165: TEST_backfill_priority: grep -q 'doesn'\''t require backfilling' 2020-09-12T14:16:01.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:166: TEST_backfill_priority: break 2020-09-12T14:16:01.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:174: TEST_backfill_priority: flush_pg_stats 2020-09-12T14:16:01.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:16:01.546 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:16:01.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:16:01.979 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:16:01.979 INFO:tasks.workunit.client.0.smithi027.stderr:2 2020-09-12T14:16:01.979 INFO:tasks.workunit.client.0.smithi027.stderr:3 2020-09-12T14:16:01.979 INFO:tasks.workunit.client.0.smithi027.stderr:4' 2020-09-12T14:16:01.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:16:01.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:01.980 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:16:02.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836523 2020-09-12T14:16:02.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836523 2020-09-12T14:16:02.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836523' 2020-09-12T14:16:02.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:02.124 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:16:02.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949673003 2020-09-12T14:16:02.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949673003 2020-09-12T14:16:02.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836523 1-42949673003' 2020-09-12T14:16:02.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:02.268 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:16:02.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509480 2020-09-12T14:16:02.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509480 2020-09-12T14:16:02.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836523 1-42949673003 2-64424509480' 2020-09-12T14:16:02.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:02.412 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2020-09-12T14:16:02.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=85899345959 2020-09-12T14:16:02.557 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 85899345959 2020-09-12T14:16:02.557 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836523 1-42949673003 2-64424509480 3-85899345959' 2020-09-12T14:16:02.557 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:02.557 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2020-09-12T14:16:02.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=107374182437 2020-09-12T14:16:02.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 107374182437 2020-09-12T14:16:02.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836523 1-42949673003 2-64424509480 3-85899345959 4-107374182437' 2020-09-12T14:16:02.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:02.702 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836523 2020-09-12T14:16:02.702 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:02.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:16:02.703 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836523 2020-09-12T14:16:02.703 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:02.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836523 2020-09-12T14:16:02.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836523' 2020-09-12T14:16:02.704 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:16:02.704 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836523 2020-09-12T14:16:03.113 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836521 -lt 21474836523 2020-09-12T14:16:03.113 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:16:04.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:16:04.115 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:16:04.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836524 -lt 21474836523 2020-09-12T14:16:04.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:04.541 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949673003 2020-09-12T14:16:04.541 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:04.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:16:04.542 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949673003 2020-09-12T14:16:04.542 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:04.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949673003 2020-09-12T14:16:04.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949673003' 2020-09-12T14:16:04.543 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949673003 2020-09-12T14:16:04.543 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:16:04.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949673003 -lt 42949673003 2020-09-12T14:16:04.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:04.955 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509480 2020-09-12T14:16:04.955 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:04.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:16:04.956 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509480 2020-09-12T14:16:04.956 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:04.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509480 2020-09-12T14:16:04.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509480' 2020-09-12T14:16:04.958 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:16:04.958 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509480 2020-09-12T14:16:05.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509480 -lt 64424509480 2020-09-12T14:16:05.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:05.369 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 3-85899345959 2020-09-12T14:16:05.370 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:05.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=3 2020-09-12T14:16:05.370 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 3-85899345959 2020-09-12T14:16:05.371 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:05.371 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=85899345959 2020-09-12T14:16:05.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.3 seq 85899345959' 2020-09-12T14:16:05.372 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.3 seq 85899345959 2020-09-12T14:16:05.372 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 3 2020-09-12T14:16:05.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 85899345959 -lt 85899345959 2020-09-12T14:16:05.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:05.788 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 4-107374182437 2020-09-12T14:16:05.788 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:05.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=4 2020-09-12T14:16:05.789 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 4-107374182437 2020-09-12T14:16:05.789 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:05.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=107374182437 2020-09-12T14:16:05.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.4 seq 107374182437' 2020-09-12T14:16:05.790 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.4 seq 107374182437 2020-09-12T14:16:05.791 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 4 2020-09-12T14:16:06.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 107374182438 -lt 107374182437 2020-09-12T14:16:06.204 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:175: TEST_backfill_priority: get_asok_path osd.1 2020-09-12T14:16:06.205 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2020-09-12T14:16:06.205 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2020-09-12T14:16:06.205 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:16:06.205 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:16:06.205 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:16:06.205 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.73258/ceph-osd.1.asok 2020-09-12T14:16:06.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:175: TEST_backfill_priority: CEPH_ARGS= 2020-09-12T14:16:06.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:175: TEST_backfill_priority: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:16:06.307 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:16:06.308 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:16:06.308 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:16:06.308 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:16:06.308 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:16:06.308 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:16:06.308 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:06.308 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "7.0", 2020-09-12T14:16:06.309 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 254, 2020-09-12T14:16:06.309 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:06.309 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:06.309 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:06.310 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:16:06.310 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:16:06.310 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:16:06.310 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:16:06.311 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:16:06.311 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:16:06.311 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:06.311 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:16:06.316 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:177: TEST_backfill_priority: ceph osd out osd.0 2020-09-12T14:16:06.870 INFO:tasks.workunit.client.0.smithi027.stderr:osd.0 is already out. 2020-09-12T14:16:06.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:178: TEST_backfill_priority: sleep 2 2020-09-12T14:16:08.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:179: TEST_backfill_priority: flush_pg_stats 2020-09-12T14:16:08.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:16:08.881 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:16:09.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:16:09.297 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:16:09.298 INFO:tasks.workunit.client.0.smithi027.stderr:2 2020-09-12T14:16:09.298 INFO:tasks.workunit.client.0.smithi027.stderr:3 2020-09-12T14:16:09.298 INFO:tasks.workunit.client.0.smithi027.stderr:4' 2020-09-12T14:16:09.298 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:16:09.298 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:09.298 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:16:09.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836527 2020-09-12T14:16:09.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836527 2020-09-12T14:16:09.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836527' 2020-09-12T14:16:09.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:09.485 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:16:09.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949673006 2020-09-12T14:16:09.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949673006 2020-09-12T14:16:09.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836527 1-42949673006' 2020-09-12T14:16:09.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:09.629 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:16:09.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509484 2020-09-12T14:16:09.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509484 2020-09-12T14:16:09.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836527 1-42949673006 2-64424509484' 2020-09-12T14:16:09.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:09.784 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2020-09-12T14:16:09.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=85899345962 2020-09-12T14:16:09.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 85899345962 2020-09-12T14:16:09.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836527 1-42949673006 2-64424509484 3-85899345962' 2020-09-12T14:16:09.928 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:09.928 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2020-09-12T14:16:10.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=107374182440 2020-09-12T14:16:10.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 107374182440 2020-09-12T14:16:10.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836527 1-42949673006 2-64424509484 3-85899345962 4-107374182440' 2020-09-12T14:16:10.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:10.072 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836527 2020-09-12T14:16:10.072 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:10.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:16:10.073 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836527 2020-09-12T14:16:10.073 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:10.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836527 2020-09-12T14:16:10.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836527' 2020-09-12T14:16:10.074 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:16:10.074 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836527 2020-09-12T14:16:10.481 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836525 -lt 21474836527 2020-09-12T14:16:10.481 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:16:11.482 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:16:11.483 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:16:11.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836527 -lt 21474836527 2020-09-12T14:16:11.932 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:11.932 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949673006 2020-09-12T14:16:11.932 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:11.933 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:16:11.933 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949673006 2020-09-12T14:16:11.933 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:11.934 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949673006 2020-09-12T14:16:11.934 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949673006' 2020-09-12T14:16:11.934 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949673006 2020-09-12T14:16:11.935 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:16:12.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949673007 -lt 42949673006 2020-09-12T14:16:12.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:12.344 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509484 2020-09-12T14:16:12.344 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:12.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:16:12.345 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509484 2020-09-12T14:16:12.345 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:12.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509484 2020-09-12T14:16:12.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509484' 2020-09-12T14:16:12.346 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509484 2020-09-12T14:16:12.347 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:16:12.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509484 -lt 64424509484 2020-09-12T14:16:12.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:12.751 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 3-85899345962 2020-09-12T14:16:12.752 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:12.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=3 2020-09-12T14:16:12.752 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 3-85899345962 2020-09-12T14:16:12.753 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:12.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=85899345962 2020-09-12T14:16:12.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.3 seq 85899345962' 2020-09-12T14:16:12.754 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.3 seq 85899345962 2020-09-12T14:16:12.754 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 3 2020-09-12T14:16:13.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 85899345962 -lt 85899345962 2020-09-12T14:16:13.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:13.158 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 4-107374182440 2020-09-12T14:16:13.158 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:13.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=4 2020-09-12T14:16:13.159 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 4-107374182440 2020-09-12T14:16:13.159 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:13.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=107374182440 2020-09-12T14:16:13.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.4 seq 107374182440' 2020-09-12T14:16:13.160 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.4 seq 107374182440 2020-09-12T14:16:13.160 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 4 2020-09-12T14:16:13.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 107374182441 -lt 107374182440 2020-09-12T14:16:13.566 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:180: TEST_backfill_priority: get_asok_path osd.1 2020-09-12T14:16:13.566 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2020-09-12T14:16:13.567 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2020-09-12T14:16:13.567 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:16:13.567 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:16:13.567 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:16:13.567 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.73258/ceph-osd.1.asok 2020-09-12T14:16:13.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:180: TEST_backfill_priority: CEPH_ARGS= 2020-09-12T14:16:13.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:180: TEST_backfill_priority: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:16:13.669 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:16:13.670 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:16:13.670 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:16:13.670 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:16:13.670 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:16:13.670 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:13.670 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:16:13.671 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:16:13.671 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:13.671 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:16:13.671 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:16:13.671 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:13.671 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:13.671 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:13.671 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:13.672 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:16:13.672 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:16:13.672 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:13.672 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "7.0", 2020-09-12T14:16:13.672 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 254, 2020-09-12T14:16:13.672 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:13.672 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:13.673 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:13.673 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:16:13.673 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:16:13.673 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:16:13.673 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:16:13.673 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:16:13.673 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:16:13.673 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:13.674 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:16:13.678 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:181: TEST_backfill_priority: ceph pg dump pgs 2020-09-12T14:16:14.058 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:16:14.059 INFO:tasks.workunit.client.0.smithi027.stdout:7.0 50 0 50 0 0 524288000 0 0 50 50 active+undersized+degraded+remapped+backfilling+forced_backfill 2020-09-12T14:16:01.417293+0000 93'150 104:214 [1,4] 1 [1] 1 0'0 2020-09-12T14:13:46.480650+0000 0'0 2020-09-12T14:13:46.480650+0000 0 2020-09-12T14:16:14.059 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 50 0 0 50 0 524288000 0 0 50 50 active+remapped+backfill_wait 2020-09-12T14:16:08.122283+0000 93'150 105:224 [1,2] 1 [1,0] 1 0'0 2020-09-12T14:13:30.255666+0000 0'0 2020-09-12T14:13:30.255666+0000 0 2020-09-12T14:16:14.059 INFO:tasks.workunit.client.0.smithi027.stdout:3.0 50 0 0 0 0 524288000 0 0 50 50 active+clean 2020-09-12T14:14:17.841130+0000 93'150 104:220 [1] 1 [1] 1 0'0 2020-09-12T14:13:35.363167+0000 0'0 2020-09-12T14:13:35.363167+0000 0 2020-09-12T14:16:14.059 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:16:14.059 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:16:14.060 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:16:14.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:183: TEST_backfill_priority: ceph osd pool set test3 size 2 2020-09-12T14:16:14.825 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 3 size to 2 2020-09-12T14:16:14.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:184: TEST_backfill_priority: sleep 2 2020-09-12T14:16:16.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:185: TEST_backfill_priority: flush_pg_stats 2020-09-12T14:16:16.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:16:16.837 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:16:17.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:16:17.265 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:16:17.265 INFO:tasks.workunit.client.0.smithi027.stderr:2 2020-09-12T14:16:17.266 INFO:tasks.workunit.client.0.smithi027.stderr:3 2020-09-12T14:16:17.266 INFO:tasks.workunit.client.0.smithi027.stderr:4' 2020-09-12T14:16:17.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:16:17.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:17.266 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:16:17.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836530 2020-09-12T14:16:17.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836530 2020-09-12T14:16:17.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836530' 2020-09-12T14:16:17.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:17.411 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:16:17.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949673010 2020-09-12T14:16:17.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949673010 2020-09-12T14:16:17.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836530 1-42949673010' 2020-09-12T14:16:17.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:17.556 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:16:17.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509487 2020-09-12T14:16:17.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509487 2020-09-12T14:16:17.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836530 1-42949673010 2-64424509487' 2020-09-12T14:16:17.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:17.700 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2020-09-12T14:16:17.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=85899345966 2020-09-12T14:16:17.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 85899345966 2020-09-12T14:16:17.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836530 1-42949673010 2-64424509487 3-85899345966' 2020-09-12T14:16:17.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:17.844 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2020-09-12T14:16:17.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=107374182444 2020-09-12T14:16:17.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 107374182444 2020-09-12T14:16:17.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836530 1-42949673010 2-64424509487 3-85899345966 4-107374182444' 2020-09-12T14:16:17.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:17.988 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836530 2020-09-12T14:16:17.988 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:17.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:16:17.989 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836530 2020-09-12T14:16:17.989 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:17.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836530 2020-09-12T14:16:17.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836530' 2020-09-12T14:16:17.990 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836530 2020-09-12T14:16:17.990 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:16:18.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836530 -lt 21474836530 2020-09-12T14:16:18.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:18.391 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949673010 2020-09-12T14:16:18.391 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:18.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:16:18.392 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949673010 2020-09-12T14:16:18.392 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:18.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949673010 2020-09-12T14:16:18.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949673010' 2020-09-12T14:16:18.393 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949673010 2020-09-12T14:16:18.394 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:16:18.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949673008 -lt 42949673010 2020-09-12T14:16:18.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:16:19.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:16:19.801 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:16:20.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949673010 -lt 42949673010 2020-09-12T14:16:20.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:20.215 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509487 2020-09-12T14:16:20.215 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:20.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:16:20.216 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509487 2020-09-12T14:16:20.216 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:20.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509487 2020-09-12T14:16:20.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509487' 2020-09-12T14:16:20.217 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509487 2020-09-12T14:16:20.218 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:16:20.621 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509488 -lt 64424509487 2020-09-12T14:16:20.621 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:20.622 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 3-85899345966 2020-09-12T14:16:20.622 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:20.622 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=3 2020-09-12T14:16:20.623 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 3-85899345966 2020-09-12T14:16:20.623 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:20.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=85899345966 2020-09-12T14:16:20.625 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.3 seq 85899345966' 2020-09-12T14:16:20.625 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 3 2020-09-12T14:16:20.625 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.3 seq 85899345966 2020-09-12T14:16:21.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 85899345966 -lt 85899345966 2020-09-12T14:16:21.038 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:21.038 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 4-107374182444 2020-09-12T14:16:21.038 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:21.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=4 2020-09-12T14:16:21.039 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 4-107374182444 2020-09-12T14:16:21.039 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:21.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=107374182444 2020-09-12T14:16:21.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.4 seq 107374182444' 2020-09-12T14:16:21.041 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.4 seq 107374182444 2020-09-12T14:16:21.041 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 4 2020-09-12T14:16:21.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 107374182444 -lt 107374182444 2020-09-12T14:16:21.541 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:186: TEST_backfill_priority: get_asok_path osd.1 2020-09-12T14:16:21.541 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2020-09-12T14:16:21.541 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2020-09-12T14:16:21.541 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:16:21.542 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:16:21.542 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:16:21.542 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.73258/ceph-osd.1.asok 2020-09-12T14:16:21.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:186: TEST_backfill_priority: CEPH_ARGS= 2020-09-12T14:16:21.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:186: TEST_backfill_priority: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:16:21.653 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:187: TEST_backfill_priority: cat td/osd-backfill-prio/out 2020-09-12T14:16:21.654 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:16:21.654 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:16:21.654 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:16:21.654 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:16:21.654 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:16:21.655 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:21.655 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:16:21.655 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:16:21.655 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:21.655 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:16:21.655 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:16:21.655 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:21.655 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:21.656 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:21.656 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:16:21.656 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:21.656 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 151, 2020-09-12T14:16:21.656 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:16:21.656 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:21.656 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:16:21.657 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:16:21.657 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:21.657 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:21.657 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:21.657 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:21.657 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:16:21.657 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:16:21.658 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:21.658 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "7.0", 2020-09-12T14:16:21.658 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 254, 2020-09-12T14:16:21.658 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:21.658 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:21.658 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:21.658 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:16:21.658 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:16:21.659 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:16:21.659 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:16:21.659 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:16:21.659 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:16:21.659 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:21.659 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:16:21.659 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:188: TEST_backfill_priority: ceph pg dump pgs 2020-09-12T14:16:22.034 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:16:22.035 INFO:tasks.workunit.client.0.smithi027.stdout:7.0 50 0 50 0 0 524288000 0 0 50 50 active+undersized+degraded+remapped+backfilling+forced_backfill 2020-09-12T14:16:01.417293+0000 93'150 109:219 [1,4] 1 [1] 1 0'0 2020-09-12T14:13:46.480650+0000 0'0 2020-09-12T14:13:46.480650+0000 0 2020-09-12T14:16:22.035 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 50 0 0 50 0 524288000 0 0 50 50 active+remapped+backfill_wait 2020-09-12T14:16:08.122283+0000 93'150 109:229 [1,2] 1 [1,0] 1 0'0 2020-09-12T14:13:30.255666+0000 0'0 2020-09-12T14:13:30.255666+0000 0 2020-09-12T14:16:22.035 INFO:tasks.workunit.client.0.smithi027.stdout:3.0 50 0 50 0 0 524288000 0 0 50 50 active+undersized+degraded+remapped+backfill_wait 2020-09-12T14:16:17.106243+0000 93'150 110:232 [1,2] 1 [1] 1 0'0 2020-09-12T14:13:35.363167+0000 0'0 2020-09-12T14:13:35.363167+0000 0 2020-09-12T14:16:22.035 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:16:22.035 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:16:22.036 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:16:22.046 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:190: TEST_backfill_priority: cat td/osd-backfill-prio/out 2020-09-12T14:16:22.047 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:190: TEST_backfill_priority: jq '(.local_reservations.queues[].items[] | select(.item == "1.0")).prio' 2020-09-12T14:16:22.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:190: TEST_backfill_priority: PRIO=110 2020-09-12T14:16:22.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:191: TEST_backfill_priority: '[' 110 '!=' 110 ']' 2020-09-12T14:16:22.049 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:198: TEST_backfill_priority: cat td/osd-backfill-prio/out 2020-09-12T14:16:22.050 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:198: TEST_backfill_priority: jq '.local_reservations.in_progress[0].item' 2020-09-12T14:16:22.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:198: TEST_backfill_priority: eval 'ITEM="7.0"' 2020-09-12T14:16:22.052 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:198: TEST_backfill_priority: ITEM=7.0 2020-09-12T14:16:22.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:199: TEST_backfill_priority: '[' 7.0 '!=' 7.0 ']' 2020-09-12T14:16:22.052 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:204: TEST_backfill_priority: cat td/osd-backfill-prio/out 2020-09-12T14:16:22.053 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:204: TEST_backfill_priority: jq '.local_reservations.in_progress[0].prio' 2020-09-12T14:16:22.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:204: TEST_backfill_priority: PRIO=254 2020-09-12T14:16:22.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:205: TEST_backfill_priority: '[' 254 '!=' 254 ']' 2020-09-12T14:16:22.055 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:213: TEST_backfill_priority: seq 1 10 2020-09-12T14:16:22.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:213: TEST_backfill_priority: for i in $(seq 1 $max_tries) 2020-09-12T14:16:22.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:215: TEST_backfill_priority: ceph pg force-backfill 3.0 2020-09-12T14:16:22.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:215: TEST_backfill_priority: grep -q 'doesn'\''t require backfilling' 2020-09-12T14:16:22.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:216: TEST_backfill_priority: break 2020-09-12T14:16:22.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:224: TEST_backfill_priority: sleep 2 2020-09-12T14:16:24.452 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:225: TEST_backfill_priority: get_asok_path osd.1 2020-09-12T14:16:24.452 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2020-09-12T14:16:24.452 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2020-09-12T14:16:24.452 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:16:24.453 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:16:24.453 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:16:24.453 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.73258/ceph-osd.1.asok 2020-09-12T14:16:24.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:225: TEST_backfill_priority: CEPH_ARGS= 2020-09-12T14:16:24.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:225: TEST_backfill_priority: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:16:24.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:226: TEST_backfill_priority: cat td/osd-backfill-prio/out 2020-09-12T14:16:24.576 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:16:24.576 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:16:24.577 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:16:24.577 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:16:24.577 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:16:24.577 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:24.577 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:16:24.577 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:16:24.577 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:24.578 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:16:24.578 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:16:24.578 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:24.578 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:24.578 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:24.578 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:16:24.578 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:24.579 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 254, 2020-09-12T14:16:24.579 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:16:24.579 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:24.579 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:16:24.579 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 254, 2020-09-12T14:16:24.579 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:24.579 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:24.580 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:24.580 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:24.580 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:16:24.580 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:16:24.580 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:24.580 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "7.0", 2020-09-12T14:16:24.580 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 254, 2020-09-12T14:16:24.581 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:24.581 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:24.581 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:24.581 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:16:24.581 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:16:24.582 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:16:24.582 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:16:24.582 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:16:24.583 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:16:24.583 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:24.583 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:16:24.584 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:227: TEST_backfill_priority: cat td/osd-backfill-prio/out 2020-09-12T14:16:24.584 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:227: TEST_backfill_priority: jq '(.local_reservations.queues[].items[] | select(.item == "3.0")).prio' 2020-09-12T14:16:24.584 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:227: TEST_backfill_priority: PRIO=254 2020-09-12T14:16:24.584 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:228: TEST_backfill_priority: '[' 254 '!=' 254 ']' 2020-09-12T14:16:24.584 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:233: TEST_backfill_priority: flush_pg_stats 2020-09-12T14:16:24.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:16:24.585 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:16:24.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:16:24.982 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:16:24.982 INFO:tasks.workunit.client.0.smithi027.stderr:2 2020-09-12T14:16:24.983 INFO:tasks.workunit.client.0.smithi027.stderr:3 2020-09-12T14:16:24.983 INFO:tasks.workunit.client.0.smithi027.stderr:4' 2020-09-12T14:16:24.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:16:24.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:24.983 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:16:25.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836534 2020-09-12T14:16:25.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836534 2020-09-12T14:16:25.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836534' 2020-09-12T14:16:25.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:25.132 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:16:25.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949673013 2020-09-12T14:16:25.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949673013 2020-09-12T14:16:25.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836534 1-42949673013' 2020-09-12T14:16:25.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:25.276 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:16:25.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509491 2020-09-12T14:16:25.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509491 2020-09-12T14:16:25.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836534 1-42949673013 2-64424509491' 2020-09-12T14:16:25.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:25.421 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2020-09-12T14:16:25.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=85899345969 2020-09-12T14:16:25.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 85899345969 2020-09-12T14:16:25.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836534 1-42949673013 2-64424509491 3-85899345969' 2020-09-12T14:16:25.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:25.566 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2020-09-12T14:16:25.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=107374182448 2020-09-12T14:16:25.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 107374182448 2020-09-12T14:16:25.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836534 1-42949673013 2-64424509491 3-85899345969 4-107374182448' 2020-09-12T14:16:25.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:25.709 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836534 2020-09-12T14:16:25.709 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:25.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:16:25.710 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836534 2020-09-12T14:16:25.710 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:25.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836534 2020-09-12T14:16:25.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836534' 2020-09-12T14:16:25.711 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:16:25.711 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836534 2020-09-12T14:16:26.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836534 -lt 21474836534 2020-09-12T14:16:26.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:26.129 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949673013 2020-09-12T14:16:26.129 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:26.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:16:26.129 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949673013 2020-09-12T14:16:26.130 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:26.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949673013 2020-09-12T14:16:26.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949673013' 2020-09-12T14:16:26.131 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949673013 2020-09-12T14:16:26.131 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:16:26.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949673013 -lt 42949673013 2020-09-12T14:16:26.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:26.543 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509491 2020-09-12T14:16:26.543 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:26.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:16:26.544 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509491 2020-09-12T14:16:26.544 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:26.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509491 2020-09-12T14:16:26.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509491' 2020-09-12T14:16:26.545 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509491 2020-09-12T14:16:26.546 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:16:26.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509491 -lt 64424509491 2020-09-12T14:16:26.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:26.978 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 3-85899345969 2020-09-12T14:16:26.978 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:26.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=3 2020-09-12T14:16:26.979 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 3-85899345969 2020-09-12T14:16:26.979 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:26.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=85899345969 2020-09-12T14:16:26.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.3 seq 85899345969' 2020-09-12T14:16:26.980 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 3 2020-09-12T14:16:26.980 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.3 seq 85899345969 2020-09-12T14:16:27.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 85899345967 -lt 85899345969 2020-09-12T14:16:27.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:16:28.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:16:28.427 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 3 2020-09-12T14:16:28.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 85899345970 -lt 85899345969 2020-09-12T14:16:28.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:28.847 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 4-107374182448 2020-09-12T14:16:28.847 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:28.848 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=4 2020-09-12T14:16:28.849 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 4-107374182448 2020-09-12T14:16:28.849 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:28.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=107374182448 2020-09-12T14:16:28.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.4 seq 107374182448' 2020-09-12T14:16:28.850 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 4 2020-09-12T14:16:28.850 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.4 seq 107374182448 2020-09-12T14:16:29.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 107374182448 -lt 107374182448 2020-09-12T14:16:29.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:236: TEST_backfill_priority: ceph pg cancel-force-backfill 7.0 2020-09-12T14:16:29.647 INFO:tasks.workunit.client.0.smithi027.stderr:instructing pg(s) [7.0] on osd.1 to cancel-force-backfill; 2020-09-12T14:16:29.647 INFO:tasks.workunit.client.0.smithi027.stderr: 2020-09-12T14:16:29.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:237: TEST_backfill_priority: sleep 2 2020-09-12T14:16:31.658 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:238: TEST_backfill_priority: get_asok_path osd.1 2020-09-12T14:16:31.658 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2020-09-12T14:16:31.659 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2020-09-12T14:16:31.659 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:16:31.659 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:16:31.659 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:16:31.659 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.73258/ceph-osd.1.asok 2020-09-12T14:16:31.660 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:238: TEST_backfill_priority: CEPH_ARGS= 2020-09-12T14:16:31.660 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:238: TEST_backfill_priority: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:16:31.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:239: TEST_backfill_priority: cat td/osd-backfill-prio/out 2020-09-12T14:16:31.783 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:16:31.783 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:16:31.784 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:16:31.784 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:16:31.784 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:16:31.784 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:31.784 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:16:31.784 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:16:31.784 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:31.785 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:16:31.785 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:16:31.785 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:31.785 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:31.785 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:31.785 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:16:31.785 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:31.785 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 151, 2020-09-12T14:16:31.786 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:16:31.786 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:31.786 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "7.0", 2020-09-12T14:16:31.786 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:16:31.786 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:31.786 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:31.786 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:31.787 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:31.787 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:16:31.787 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:16:31.787 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:31.787 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:16:31.787 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 254, 2020-09-12T14:16:31.787 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:31.787 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:31.788 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:31.788 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:16:31.788 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:16:31.788 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:16:31.788 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:16:31.788 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:16:31.788 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:16:31.789 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:31.789 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:16:31.790 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:240: TEST_backfill_priority: cat td/osd-backfill-prio/out 2020-09-12T14:16:31.790 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:240: TEST_backfill_priority: jq '(.local_reservations.queues[].items[] | select(.item == "7.0")).prio' 2020-09-12T14:16:31.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:240: TEST_backfill_priority: PRIO=151 2020-09-12T14:16:31.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:241: TEST_backfill_priority: '[' 151 '!=' 151 ']' 2020-09-12T14:16:31.791 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:247: TEST_backfill_priority: cat td/osd-backfill-prio/out 2020-09-12T14:16:31.791 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:247: TEST_backfill_priority: jq '.local_reservations.in_progress[0].item' 2020-09-12T14:16:31.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:247: TEST_backfill_priority: eval 'ITEM="3.0"' 2020-09-12T14:16:31.791 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:247: TEST_backfill_priority: ITEM=3.0 2020-09-12T14:16:31.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:248: TEST_backfill_priority: '[' 3.0 '!=' 3.0 ']' 2020-09-12T14:16:31.792 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:253: TEST_backfill_priority: cat td/osd-backfill-prio/out 2020-09-12T14:16:31.792 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:253: TEST_backfill_priority: jq '.local_reservations.in_progress[0].prio' 2020-09-12T14:16:31.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:253: TEST_backfill_priority: PRIO=254 2020-09-12T14:16:31.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:254: TEST_backfill_priority: '[' 254 '!=' 254 ']' 2020-09-12T14:16:31.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:261: TEST_backfill_priority: ceph pg cancel-force-backfill 3.0 2020-09-12T14:16:32.182 INFO:tasks.workunit.client.0.smithi027.stderr:instructing pg(s) [3.0] on osd.1 to cancel-force-backfill; 2020-09-12T14:16:32.182 INFO:tasks.workunit.client.0.smithi027.stderr: 2020-09-12T14:16:32.191 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:262: TEST_backfill_priority: sleep 5 2020-09-12T14:16:37.193 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:263: TEST_backfill_priority: get_asok_path osd.1 2020-09-12T14:16:37.193 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2020-09-12T14:16:37.194 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2020-09-12T14:16:37.194 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:16:37.194 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:16:37.194 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:16:37.195 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.73258/ceph-osd.1.asok 2020-09-12T14:16:37.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:263: TEST_backfill_priority: CEPH_ARGS= 2020-09-12T14:16:37.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:263: TEST_backfill_priority: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:16:37.309 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:16:37.309 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:16:37.309 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:16:37.309 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:16:37.310 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:16:37.310 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:37.310 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:16:37.310 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:16:37.311 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:37.311 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:16:37.311 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:16:37.311 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:37.312 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:37.312 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:37.312 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:16:37.312 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:37.313 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 151, 2020-09-12T14:16:37.313 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:16:37.313 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:37.313 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "7.0", 2020-09-12T14:16:37.314 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:16:37.314 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:37.314 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:37.314 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:37.315 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:37.316 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:16:37.316 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:16:37.316 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:37.317 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:16:37.317 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:16:37.317 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:37.317 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:37.317 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:37.318 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:16:37.318 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:16:37.318 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:16:37.318 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:16:37.318 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:16:37.319 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:16:37.319 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:37.319 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:16:37.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:266: TEST_backfill_priority: flush_pg_stats 2020-09-12T14:16:37.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:16:37.320 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:16:37.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:16:37.730 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:16:37.730 INFO:tasks.workunit.client.0.smithi027.stderr:2 2020-09-12T14:16:37.730 INFO:tasks.workunit.client.0.smithi027.stderr:3 2020-09-12T14:16:37.730 INFO:tasks.workunit.client.0.smithi027.stderr:4' 2020-09-12T14:16:37.730 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:16:37.730 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:37.731 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:16:37.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836538 2020-09-12T14:16:37.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836538 2020-09-12T14:16:37.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836538' 2020-09-12T14:16:37.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:37.874 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:16:38.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949673018 2020-09-12T14:16:38.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949673018 2020-09-12T14:16:38.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836538 1-42949673018' 2020-09-12T14:16:38.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:38.018 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:16:38.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509495 2020-09-12T14:16:38.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509495 2020-09-12T14:16:38.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836538 1-42949673018 2-64424509495' 2020-09-12T14:16:38.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:38.162 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2020-09-12T14:16:38.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=85899345974 2020-09-12T14:16:38.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 85899345974 2020-09-12T14:16:38.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836538 1-42949673018 2-64424509495 3-85899345974' 2020-09-12T14:16:38.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:38.305 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2020-09-12T14:16:38.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=107374182452 2020-09-12T14:16:38.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 107374182452 2020-09-12T14:16:38.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836538 1-42949673018 2-64424509495 3-85899345974 4-107374182452' 2020-09-12T14:16:38.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:38.449 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836538 2020-09-12T14:16:38.449 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:38.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:16:38.450 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836538 2020-09-12T14:16:38.450 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:38.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836538 2020-09-12T14:16:38.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836538' 2020-09-12T14:16:38.451 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:16:38.451 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836538 2020-09-12T14:16:38.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836536 -lt 21474836538 2020-09-12T14:16:38.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:16:39.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:16:39.854 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:16:40.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836539 -lt 21474836538 2020-09-12T14:16:40.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:40.273 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949673018 2020-09-12T14:16:40.273 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:40.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:16:40.274 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949673018 2020-09-12T14:16:40.274 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:40.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949673018 2020-09-12T14:16:40.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949673018' 2020-09-12T14:16:40.275 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949673018 2020-09-12T14:16:40.275 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:16:40.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949673018 -lt 42949673018 2020-09-12T14:16:40.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:40.714 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509495 2020-09-12T14:16:40.714 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:40.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:16:40.715 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509495 2020-09-12T14:16:40.715 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:40.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509495 2020-09-12T14:16:40.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509495' 2020-09-12T14:16:40.716 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509495 2020-09-12T14:16:40.717 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:16:41.119 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509496 -lt 64424509495 2020-09-12T14:16:41.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:41.121 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 3-85899345974 2020-09-12T14:16:41.121 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:41.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=3 2020-09-12T14:16:41.123 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 3-85899345974 2020-09-12T14:16:41.123 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:41.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=85899345974 2020-09-12T14:16:41.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.3 seq 85899345974' 2020-09-12T14:16:41.124 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 3 2020-09-12T14:16:41.124 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.3 seq 85899345974 2020-09-12T14:16:41.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 85899345974 -lt 85899345974 2020-09-12T14:16:41.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:41.570 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 4-107374182452 2020-09-12T14:16:41.570 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:41.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=4 2020-09-12T14:16:41.571 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 4-107374182452 2020-09-12T14:16:41.571 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:41.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=107374182452 2020-09-12T14:16:41.573 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.4 seq 107374182452' 2020-09-12T14:16:41.573 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 4 2020-09-12T14:16:41.573 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.4 seq 107374182452 2020-09-12T14:16:41.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 107374182453 -lt 107374182452 2020-09-12T14:16:41.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:267: TEST_backfill_priority: ceph pg force-backfill 7.0 2020-09-12T14:16:42.370 INFO:tasks.workunit.client.0.smithi027.stderr:instructing pg(s) [7.0] on osd.1 to force-backfill; 2020-09-12T14:16:42.370 INFO:tasks.workunit.client.0.smithi027.stderr: 2020-09-12T14:16:42.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:268: TEST_backfill_priority: sleep 2 2020-09-12T14:16:44.381 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:270: TEST_backfill_priority: get_asok_path osd.1 2020-09-12T14:16:44.382 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2020-09-12T14:16:44.382 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2020-09-12T14:16:44.382 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:16:44.382 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:16:44.383 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:16:44.383 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.73258/ceph-osd.1.asok 2020-09-12T14:16:44.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:270: TEST_backfill_priority: CEPH_ARGS= 2020-09-12T14:16:44.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:270: TEST_backfill_priority: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:16:44.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:271: TEST_backfill_priority: cat td/osd-backfill-prio/out 2020-09-12T14:16:44.505 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:16:44.506 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:16:44.506 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:16:44.506 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:16:44.506 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:16:44.506 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:44.506 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:16:44.506 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:16:44.507 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:44.507 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:16:44.507 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:16:44.507 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:44.507 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:44.507 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:44.507 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:16:44.508 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:44.508 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 151, 2020-09-12T14:16:44.508 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:16:44.508 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:44.508 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:16:44.508 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:16:44.508 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:44.508 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:44.509 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:44.509 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:44.509 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:16:44.509 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:16:44.509 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:44.509 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "7.0", 2020-09-12T14:16:44.509 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 254, 2020-09-12T14:16:44.509 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:44.510 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:44.510 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:44.510 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:16:44.510 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:16:44.510 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:16:44.510 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:16:44.510 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:16:44.511 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:16:44.511 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:44.511 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:16:44.512 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:272: TEST_backfill_priority: cat td/osd-backfill-prio/out 2020-09-12T14:16:44.512 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:272: TEST_backfill_priority: jq '(.local_reservations.queues[].items[] | select(.item == "3.0")).prio' 2020-09-12T14:16:44.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:272: TEST_backfill_priority: PRIO=151 2020-09-12T14:16:44.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:273: TEST_backfill_priority: '[' 151 '!=' 151 ']' 2020-09-12T14:16:44.513 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:279: TEST_backfill_priority: cat td/osd-backfill-prio/out 2020-09-12T14:16:44.513 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:279: TEST_backfill_priority: jq '.local_reservations.in_progress[0].item' 2020-09-12T14:16:44.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:279: TEST_backfill_priority: eval 'ITEM="7.0"' 2020-09-12T14:16:44.513 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:279: TEST_backfill_priority: ITEM=7.0 2020-09-12T14:16:44.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:280: TEST_backfill_priority: '[' 7.0 '!=' 7.0 ']' 2020-09-12T14:16:44.513 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:285: TEST_backfill_priority: cat td/osd-backfill-prio/out 2020-09-12T14:16:44.513 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:285: TEST_backfill_priority: jq '.local_reservations.in_progress[0].prio' 2020-09-12T14:16:44.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:285: TEST_backfill_priority: PRIO=254 2020-09-12T14:16:44.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:286: TEST_backfill_priority: '[' 254 '!=' 254 ']' 2020-09-12T14:16:44.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:293: TEST_backfill_priority: ceph osd unset noout 2020-09-12T14:16:45.289 INFO:tasks.workunit.client.0.smithi027.stderr:noout is unset 2020-09-12T14:16:45.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:294: TEST_backfill_priority: ceph osd unset nobackfill 2020-09-12T14:16:46.235 INFO:tasks.workunit.client.0.smithi027.stderr:nobackfill is unset 2020-09-12T14:16:46.245 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:296: TEST_backfill_priority: get_asok_path osd.1 2020-09-12T14:16:46.245 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2020-09-12T14:16:46.246 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2020-09-12T14:16:46.246 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:16:46.246 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:16:46.246 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:16:46.246 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.73258/ceph-osd.1.asok 2020-09-12T14:16:46.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:296: TEST_backfill_priority: wait_for_clean 'CEPH_ARGS='\'''\'' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations' 2020-09-12T14:16:46.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local 'cmd=CEPH_ARGS='\'''\'' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations' 2020-09-12T14:16:46.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:16:46.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:16:46.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:16:46.248 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 90 .1 2020-09-12T14:16:46.248 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:16:46.248 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:16:46.248 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:16:46.249 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:16:46.249 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:16:46.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:16:46.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:16:46.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:16:46.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:16:46.341 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:16:46.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:16:46.748 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:16:46.748 INFO:tasks.workunit.client.0.smithi027.stderr:2 2020-09-12T14:16:46.748 INFO:tasks.workunit.client.0.smithi027.stderr:3 2020-09-12T14:16:46.748 INFO:tasks.workunit.client.0.smithi027.stderr:4' 2020-09-12T14:16:46.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:16:46.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:46.749 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:16:46.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836542 2020-09-12T14:16:46.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836542 2020-09-12T14:16:46.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836542' 2020-09-12T14:16:46.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:46.952 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:16:47.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949673022 2020-09-12T14:16:47.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949673022 2020-09-12T14:16:47.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836542 1-42949673022' 2020-09-12T14:16:47.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:47.096 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:16:47.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509499 2020-09-12T14:16:47.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509499 2020-09-12T14:16:47.240 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836542 1-42949673022 2-64424509499' 2020-09-12T14:16:47.240 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:47.240 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2020-09-12T14:16:47.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=85899345978 2020-09-12T14:16:47.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 85899345978 2020-09-12T14:16:47.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836542 1-42949673022 2-64424509499 3-85899345978' 2020-09-12T14:16:47.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:16:47.383 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2020-09-12T14:16:47.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=107374182456 2020-09-12T14:16:47.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 107374182456 2020-09-12T14:16:47.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836542 1-42949673022 2-64424509499 3-85899345978 4-107374182456' 2020-09-12T14:16:47.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:47.571 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836542 2020-09-12T14:16:47.571 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:47.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:16:47.572 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836542 2020-09-12T14:16:47.572 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:47.573 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836542 2020-09-12T14:16:47.573 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836542' 2020-09-12T14:16:47.573 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836542 2020-09-12T14:16:47.574 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:16:47.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836542 -lt 21474836542 2020-09-12T14:16:47.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:47.978 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949673022 2020-09-12T14:16:47.978 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:47.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:16:47.979 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949673022 2020-09-12T14:16:47.979 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:47.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949673022 2020-09-12T14:16:47.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949673022' 2020-09-12T14:16:47.980 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:16:47.980 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949673022 2020-09-12T14:16:48.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949673022 -lt 42949673022 2020-09-12T14:16:48.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:48.384 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509499 2020-09-12T14:16:48.385 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:48.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:16:48.386 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509499 2020-09-12T14:16:48.386 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:48.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509499 2020-09-12T14:16:48.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509499' 2020-09-12T14:16:48.387 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509499 2020-09-12T14:16:48.387 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:16:48.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509499 -lt 64424509499 2020-09-12T14:16:48.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:48.802 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 3-85899345978 2020-09-12T14:16:48.802 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:48.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=3 2020-09-12T14:16:48.803 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 3-85899345978 2020-09-12T14:16:48.803 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:48.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=85899345978 2020-09-12T14:16:48.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.3 seq 85899345978' 2020-09-12T14:16:48.804 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.3 seq 85899345978 2020-09-12T14:16:48.805 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 3 2020-09-12T14:16:49.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 85899345978 -lt 85899345978 2020-09-12T14:16:49.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:16:49.243 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 4-107374182456 2020-09-12T14:16:49.243 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:16:49.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=4 2020-09-12T14:16:49.244 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 4-107374182456 2020-09-12T14:16:49.244 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:16:49.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=107374182456 2020-09-12T14:16:49.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.4 seq 107374182456' 2020-09-12T14:16:49.245 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.4 seq 107374182456 2020-09-12T14:16:49.246 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 4 2020-09-12T14:16:49.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 107374182454 -lt 107374182456 2020-09-12T14:16:49.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:16:50.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:16:50.686 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 4 2020-09-12T14:16:51.165 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 107374182456 -lt 107374182456 2020-09-12T14:16:51.166 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:16:51.166 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:16:51.166 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:16:51.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 3 == 0 2020-09-12T14:16:51.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:16:51.738 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:16:51.738 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:16:51.739 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:16:51.739 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:16:51.739 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:16:51.739 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:16:52.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:16:52.137 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:16:52.138 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:16:52.138 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:16:52.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 3 2020-09-12T14:16:52.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' -1 2020-09-12T14:16:52.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1598: wait_for_clean: loop=0 2020-09-12T14:16:52.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1599: wait_for_clean: num_active_clean=0 2020-09-12T14:16:52.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:16:52.676 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:16:52.676 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:16:52.777 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:16:52.778 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:16:52.778 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:16:52.778 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:16:52.778 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:16:52.778 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:52.778 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:16:52.779 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:16:52.779 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:52.779 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:16:52.779 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:16:52.779 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:52.779 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:52.779 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:52.780 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:16:52.780 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:52.780 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 151, 2020-09-12T14:16:52.780 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:16:52.780 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:52.780 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:16:52.780 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:16:52.780 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:52.781 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:52.781 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:52.781 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:52.781 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:16:52.781 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:16:52.781 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:52.781 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "7.0", 2020-09-12T14:16:52.782 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 254, 2020-09-12T14:16:52.782 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:52.782 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:52.782 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:52.782 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:16:52.782 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:16:52.782 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:16:52.782 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:16:52.783 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:16:52.783 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:16:52.783 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:52.783 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:16:52.785 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:16:52.887 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:16:52.887 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:16:52.888 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:16:52.888 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:16:52.888 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:16:52.888 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:16:52.889 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:16:52.889 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:16:53.284 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:16:53.284 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:16:53.284 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:16:53.285 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:16:53.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 3 2020-09-12T14:16:53.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:16:53.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:16:53.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:16:53.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:16:53.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:16:53.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:16:53.820 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:16:53.820 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:16:54.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=2620995 2020-09-12T14:16:54.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 2620995 '!=' null 2020-09-12T14:16:54.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:16:54.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:16:54.405 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:16:54.405 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:16:54.507 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:16:54.507 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:16:54.507 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:16:54.508 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:16:54.508 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:16:54.508 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:54.508 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:16:54.508 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:16:54.508 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:54.508 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:16:54.508 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:16:54.509 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:54.509 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:54.509 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:54.509 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:16:54.509 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:54.509 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 151, 2020-09-12T14:16:54.509 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:16:54.510 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:54.510 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:16:54.510 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:16:54.510 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:54.510 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:54.510 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:54.510 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:54.510 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:16:54.511 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:16:54.511 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:54.511 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "7.0", 2020-09-12T14:16:54.511 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 254, 2020-09-12T14:16:54.511 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:54.511 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:54.511 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:54.512 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:16:54.512 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:16:54.512 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:16:54.512 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:16:54.512 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:16:54.512 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:16:54.512 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:54.512 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:16:54.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:16:54.617 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:16:54.617 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:16:54.618 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:16:54.618 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:16:54.618 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:16:54.618 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:16:54.618 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:16:54.619 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:16:55.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:16:55.018 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:16:55.019 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:16:55.019 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:16:55.671 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 3 2020-09-12T14:16:55.672 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:16:55.672 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:16:55.672 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:16:55.672 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:16:55.672 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:16:55.673 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:16:55.673 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:16:55.673 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:16:56.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=2200812 2020-09-12T14:16:56.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 2200812 '!=' null 2020-09-12T14:16:56.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:16:56.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:16:56.282 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:16:56.283 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:16:56.350 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:16:56.351 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:16:56.351 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:16:56.351 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:16:56.351 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:16:56.351 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:56.351 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:16:56.352 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:16:56.352 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:56.352 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:16:56.352 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:16:56.352 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:56.352 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:56.352 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:56.353 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:16:56.353 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:56.353 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 151, 2020-09-12T14:16:56.353 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:16:56.353 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:56.353 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:16:56.353 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:16:56.353 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:56.354 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:56.354 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:56.354 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:56.354 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:16:56.354 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:16:56.354 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:56.354 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "7.0", 2020-09-12T14:16:56.355 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 254, 2020-09-12T14:16:56.355 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:56.355 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:56.355 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:56.355 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:16:56.355 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:16:56.355 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:16:56.355 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:16:56.356 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:16:56.356 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:16:56.356 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:56.356 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:16:56.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:16:56.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:16:56.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:16:56.461 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:16:56.461 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:16:56.461 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:16:56.462 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:16:56.462 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:16:56.462 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:16:56.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:16:56.861 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:16:56.861 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:16:56.861 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:16:57.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 3 2020-09-12T14:16:57.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:16:57.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:16:57.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:16:57.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:16:57.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:16:57.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:16:57.540 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:16:57.540 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:16:58.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=10303195 2020-09-12T14:16:58.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 10303195 '!=' null 2020-09-12T14:16:58.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:16:58.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:16:58.177 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:16:58.177 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:16:58.280 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:16:58.280 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:16:58.280 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:16:58.280 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:16:58.280 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:16:58.281 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:58.281 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:16:58.281 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:16:58.281 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:58.281 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:16:58.281 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:16:58.281 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:58.282 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:58.282 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:58.282 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:16:58.282 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:58.282 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 151, 2020-09-12T14:16:58.282 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:16:58.282 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:58.282 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:16:58.283 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:16:58.283 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:58.283 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:58.283 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:58.283 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:58.283 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:16:58.283 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:16:58.284 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:16:58.284 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "7.0", 2020-09-12T14:16:58.284 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 254, 2020-09-12T14:16:58.284 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:16:58.284 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:58.284 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:16:58.284 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:16:58.284 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:16:58.285 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:16:58.285 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:16:58.285 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:16:58.285 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:16:58.285 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:16:58.285 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:16:58.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:16:58.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:16:58.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:16:58.391 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:16:58.391 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:16:58.391 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:16:58.391 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:16:58.391 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:16:58.391 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:16:58.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:16:58.832 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:16:58.832 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:16:58.833 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:16:59.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 3 2020-09-12T14:16:59.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:16:59.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:16:59.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:16:59.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:16:59.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:16:59.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:16:59.384 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:16:59.384 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:00.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=9610211 2020-09-12T14:17:00.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 9610211 '!=' null 2020-09-12T14:17:00.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:00.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:00.156 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:00.156 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:00.258 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:00.258 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:00.258 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:00.259 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:00.259 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:00.259 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:00.259 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:00.259 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:00.259 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:00.259 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:00.260 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:00.260 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:00.260 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:00.260 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:00.260 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:00.260 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:00.260 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 151, 2020-09-12T14:17:00.261 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:00.261 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:00.261 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:00.261 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:00.261 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:00.261 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:00.261 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:00.261 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:00.262 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:00.262 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:00.262 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:00.262 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "7.0", 2020-09-12T14:17:00.262 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 254, 2020-09-12T14:17:00.262 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:00.262 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:00.262 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:00.263 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:00.263 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:00.263 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:00.263 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:00.263 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:00.263 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:00.263 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:00.264 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:00.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:00.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:00.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:00.368 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:00.369 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:00.369 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:00.369 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:00.369 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:00.369 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:00.762 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:17:00.762 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:00.763 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:00.763 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:01.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 3 2020-09-12T14:17:01.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:17:01.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:01.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:01.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:01.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:01.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:01.305 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:01.305 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:01.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=16599472 2020-09-12T14:17:01.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 16599472 '!=' null 2020-09-12T14:17:01.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:01.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:01.857 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:01.857 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:01.961 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:01.961 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:01.961 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:01.962 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:01.962 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:01.962 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:01.962 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:01.962 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:01.962 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:01.962 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:01.963 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:01.963 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:01.963 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:01.963 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:01.963 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:01.963 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:01.963 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 151, 2020-09-12T14:17:01.963 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:01.964 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:01.964 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:01.964 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:01.964 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:01.964 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:01.964 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:01.964 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:01.964 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:01.965 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:01.965 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:01.965 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "7.0", 2020-09-12T14:17:01.965 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 254, 2020-09-12T14:17:01.965 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:01.965 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:01.965 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:01.966 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:01.966 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:01.966 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:01.966 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:01.966 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:01.966 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:01.966 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:01.966 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:01.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:02.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:02.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:02.072 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:02.072 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:02.072 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:02.072 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:02.072 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:02.072 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:02.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:17:02.478 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:02.479 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:02.479 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:03.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 3 2020-09-12T14:17:03.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:17:03.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:03.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:03.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:03.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:03.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:03.305 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:03.305 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:03.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=14852165 2020-09-12T14:17:03.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 14852165 '!=' null 2020-09-12T14:17:03.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:03.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:03.844 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:03.844 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:03.949 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:03.949 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:03.949 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:03.949 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:03.949 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:03.950 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:03.950 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:03.950 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:03.950 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:03.950 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:03.950 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:03.951 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:03.951 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:03.951 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:03.951 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:03.951 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:03.951 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 151, 2020-09-12T14:17:03.951 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:03.952 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:03.952 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:03.952 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:03.952 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:03.952 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:03.952 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:03.952 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:03.952 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:03.953 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:03.953 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:03.953 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "7.0", 2020-09-12T14:17:03.953 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 254, 2020-09-12T14:17:03.953 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:03.953 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:03.953 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:03.954 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:03.954 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:03.954 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:03.954 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:03.954 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:03.954 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:03.954 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:03.955 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:03.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:04.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:04.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:04.060 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:04.060 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:04.060 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:04.060 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:04.060 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:04.060 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:04.454 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:17:04.454 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:04.455 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:04.455 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:05.083 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 3 2020-09-12T14:17:05.084 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:17:05.084 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:05.084 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:05.084 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:05.084 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:05.085 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:05.085 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:05.085 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:06.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=14852192 2020-09-12T14:17:06.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 14852192 '!=' null 2020-09-12T14:17:06.231 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:06.231 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:06.231 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:06.231 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:06.343 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:06.343 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:06.344 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:06.344 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:06.344 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:06.345 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:06.345 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:06.345 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:06.345 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:06.345 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:06.346 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:06.346 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:06.346 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:06.346 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:06.347 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:06.347 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:06.347 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 151, 2020-09-12T14:17:06.347 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:06.348 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:06.348 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:06.348 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:06.348 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:06.349 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:06.349 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:06.349 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:06.349 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:06.349 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:06.350 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:06.350 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "7.0", 2020-09-12T14:17:06.350 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 254, 2020-09-12T14:17:06.350 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:06.350 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:06.350 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:06.351 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:06.351 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:06.351 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:06.351 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:06.352 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:06.352 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:06.352 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:06.352 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:06.353 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:06.454 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:06.454 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:06.455 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:06.455 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:06.455 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:06.455 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:06.456 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:06.456 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:06.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:17:06.853 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:06.854 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:06.854 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:07.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 3 2020-09-12T14:17:07.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:17:07.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:07.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:07.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:07.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:07.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:07.414 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:07.414 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:07.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=21841687 2020-09-12T14:17:07.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 21841687 '!=' null 2020-09-12T14:17:07.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:07.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:07.958 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:07.959 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:08.060 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:08.060 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:08.060 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:08.060 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:08.060 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:08.061 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:08.061 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:08.061 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:08.061 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:08.061 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:08.061 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:08.061 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:08.062 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:08.062 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:08.062 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:08.062 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:08.062 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 151, 2020-09-12T14:17:08.062 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:08.062 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:08.062 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:08.063 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:08.063 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:08.063 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:08.063 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:08.063 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:08.063 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:08.063 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:08.063 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:08.064 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "7.0", 2020-09-12T14:17:08.064 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 254, 2020-09-12T14:17:08.064 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:08.064 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:08.064 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:08.064 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:08.064 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:08.065 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:08.065 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:08.065 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:08.065 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:08.065 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:08.065 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:08.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:08.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:08.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:08.171 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:08.171 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:08.171 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:08.172 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:08.172 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:08.172 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:08.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:17:08.569 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:08.569 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:08.570 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:09.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 3 2020-09-12T14:17:09.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:17:09.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:09.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:09.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:09.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:09.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:09.289 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:09.289 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:09.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=13978636 2020-09-12T14:17:09.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 13978636 '!=' null 2020-09-12T14:17:09.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:09.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:09.821 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:09.821 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:09.925 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:09.925 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:09.925 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:09.926 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:09.926 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:09.926 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:09.926 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:09.926 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:09.926 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:09.927 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:09.927 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:09.927 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:09.927 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:09.927 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:09.927 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:09.927 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:09.927 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 151, 2020-09-12T14:17:09.928 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:09.928 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:09.928 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:09.928 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:09.928 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:09.928 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:09.928 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:09.928 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:09.929 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:09.929 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:09.929 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:09.929 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "7.0", 2020-09-12T14:17:09.929 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 254, 2020-09-12T14:17:09.929 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:09.929 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:09.930 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:09.930 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:09.930 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:09.930 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:09.930 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:09.930 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:09.930 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:09.930 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:09.931 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:09.933 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:10.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:10.035 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:10.035 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:10.036 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:10.036 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:10.036 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:10.036 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:10.036 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:10.431 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:17:10.431 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:10.432 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:10.432 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:10.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 3 2020-09-12T14:17:10.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:17:10.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:10.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:10.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:10.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:10.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:10.969 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:10.969 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:11.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=13978636 2020-09-12T14:17:11.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 13978636 '!=' null 2020-09-12T14:17:11.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:11.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:11.531 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:11.531 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:11.633 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:11.633 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:11.633 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:11.633 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:11.634 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:11.634 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:11.634 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:11.634 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:11.634 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:11.634 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:11.634 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:11.635 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:11.635 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:11.635 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:11.635 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:11.635 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:11.635 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 151, 2020-09-12T14:17:11.635 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:11.636 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:11.636 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:11.636 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:11.636 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:11.636 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:11.636 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:11.636 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:11.636 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:11.637 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:11.637 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:11.637 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "7.0", 2020-09-12T14:17:11.637 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 254, 2020-09-12T14:17:11.637 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:11.637 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:11.637 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:11.637 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:11.638 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:11.638 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:11.638 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:11.638 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:11.638 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:11.638 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:11.638 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:11.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:11.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:11.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:11.750 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:11.750 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:11.750 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:11.751 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:11.751 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:11.751 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:12.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:17:12.139 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:12.140 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:12.140 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:12.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 3 2020-09-12T14:17:12.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:17:12.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:12.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:12.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:12.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:12.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:12.738 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:12.738 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:13.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=20967957 2020-09-12T14:17:13.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 20967957 '!=' null 2020-09-12T14:17:13.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:13.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:13.322 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:13.322 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:13.424 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:13.424 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:13.425 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:13.425 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:13.425 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:13.425 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:13.425 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:13.425 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:13.425 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:13.426 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:13.426 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:13.426 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:13.426 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:13.426 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:13.426 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:13.427 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:13.427 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 151, 2020-09-12T14:17:13.427 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:13.427 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:13.427 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:13.427 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:13.427 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:13.428 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:13.428 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:13.428 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:13.428 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:13.428 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:13.428 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:13.428 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "7.0", 2020-09-12T14:17:13.428 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 254, 2020-09-12T14:17:13.429 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:13.429 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:13.429 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:13.429 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:13.429 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:13.429 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:13.429 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:13.430 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:13.430 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:13.430 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:13.430 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:13.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:13.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:13.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:13.535 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:13.535 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:13.535 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:13.535 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:13.535 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:13.536 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:14.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:17:14.019 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:14.019 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:14.020 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:14.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 3 2020-09-12T14:17:14.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:17:14.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:14.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:14.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:14.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:14.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:14.560 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:14.561 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:15.106 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=13978632 2020-09-12T14:17:15.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 13978632 '!=' null 2020-09-12T14:17:15.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:15.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:15.107 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:15.107 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:15.209 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:15.209 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:15.209 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:15.209 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:15.209 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:15.209 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:15.210 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:15.210 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:15.210 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:15.210 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:15.210 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:15.210 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:15.210 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:15.211 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:15.211 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:15.211 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:15.211 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 151, 2020-09-12T14:17:15.211 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:15.211 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:15.211 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:15.211 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:15.212 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:15.212 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:15.212 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:15.212 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:15.212 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:15.212 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:15.212 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:15.212 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "7.0", 2020-09-12T14:17:15.213 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 254, 2020-09-12T14:17:15.213 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:15.213 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:15.213 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:15.213 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:15.213 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:15.213 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:15.214 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:15.214 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:15.214 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:15.214 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:15.214 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:15.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:15.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:15.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:15.319 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:15.319 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:15.320 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:15.320 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:15.320 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:15.320 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:15.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:17:15.721 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:15.721 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:15.722 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:16.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 3 2020-09-12T14:17:16.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:17:16.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:16.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:16.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:16.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:16.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:16.255 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:16.255 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:16.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=13978645 2020-09-12T14:17:16.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 13978645 '!=' null 2020-09-12T14:17:16.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:16.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:16.790 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:16.790 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:16.892 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:16.893 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:16.893 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:16.893 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:16.893 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:16.893 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:16.894 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:16.894 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:16.894 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:16.894 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:16.894 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:16.894 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:16.894 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:16.894 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:16.895 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:16.895 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:16.895 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 151, 2020-09-12T14:17:16.895 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:16.895 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:16.895 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:16.895 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:16.896 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:16.896 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:16.896 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:16.896 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:16.896 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:16.896 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:16.896 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:16.896 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "7.0", 2020-09-12T14:17:16.897 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 254, 2020-09-12T14:17:16.897 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:16.897 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:16.897 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:16.897 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:16.897 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:16.897 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:16.897 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:16.898 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:16.898 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:16.898 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:16.898 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:16.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:17.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:17.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:17.003 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:17.003 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:17.003 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:17.004 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:17.004 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:17.004 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:17.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:17:17.400 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:17.401 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:17.401 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:18.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 3 2020-09-12T14:17:18.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:17:18.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:18.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:18.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:18.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:18.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:18.041 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:18.041 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:18.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=20967949 2020-09-12T14:17:18.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 20967949 '!=' null 2020-09-12T14:17:18.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:18.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:18.708 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:18.708 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:18.810 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:18.810 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:18.810 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:18.810 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:18.810 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:18.811 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:18.811 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:18.811 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:18.811 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:18.811 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:18.812 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:18.812 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:18.812 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:18.812 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:18.812 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:18.812 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:18.812 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 151, 2020-09-12T14:17:18.813 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:18.813 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:18.813 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:18.813 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:18.813 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:18.813 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:18.813 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:18.814 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:18.814 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:18.814 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:18.814 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:18.814 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "7.0", 2020-09-12T14:17:18.814 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 254, 2020-09-12T14:17:18.814 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:18.814 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:18.815 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:18.815 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:18.815 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:18.815 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:18.815 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:18.815 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:18.815 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:18.815 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:18.816 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:18.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:18.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:18.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:18.920 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:18.920 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:18.920 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:18.920 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:18.921 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:18.921 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:19.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:17:19.319 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:19.320 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:19.320 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:19.859 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 3 2020-09-12T14:17:19.859 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:17:19.859 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:19.859 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:19.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:19.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:19.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:19.860 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:19.860 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:20.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=13978492 2020-09-12T14:17:20.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 13978492 '!=' null 2020-09-12T14:17:20.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:20.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:20.463 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:20.464 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:20.565 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:20.566 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:20.566 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:20.566 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:20.566 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:20.566 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:20.566 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:20.566 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:20.567 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:20.567 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:20.567 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:20.567 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:20.567 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:20.567 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:20.567 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:20.568 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:20.568 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:20.568 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:20.568 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:20.568 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:20.568 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:20.568 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:20.568 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:20.569 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:20.569 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:20.569 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:20.569 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:20.569 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:20.569 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:20.569 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:20.569 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:20.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:20.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:20.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:20.677 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:20.677 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:20.677 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:20.677 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:20.678 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:20.678 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:21.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:17:21.373 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:21.373 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:21.373 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:21.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 3 2020-09-12T14:17:21.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:17:21.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:21.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:21.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:21.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:21.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:21.912 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:21.912 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:22.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=19657310 2020-09-12T14:17:22.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 19657310 '!=' null 2020-09-12T14:17:22.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:22.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:22.505 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:22.505 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:22.607 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:22.608 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:22.608 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:22.608 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:22.608 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:22.608 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:22.608 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:22.609 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:22.609 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:22.609 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:22.609 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:22.609 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:22.609 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:22.609 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:22.609 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:22.610 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:22.610 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:22.610 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:22.610 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:22.610 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:22.610 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:22.610 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:22.611 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:22.611 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:22.611 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:22.611 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:22.611 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:22.611 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:22.611 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:22.611 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:22.612 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:22.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:22.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:22.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:22.718 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:22.718 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:22.718 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:22.718 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:22.719 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:22.719 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:23.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:17:23.118 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:23.118 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:23.118 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:23.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 3 2020-09-12T14:17:23.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:17:23.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:23.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:23.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:23.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:23.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:23.712 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:23.712 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:24.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=19657258 2020-09-12T14:17:24.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 19657258 '!=' null 2020-09-12T14:17:24.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:24.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:24.256 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:24.256 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:24.357 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:24.357 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:24.357 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:24.358 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:24.358 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:24.358 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:24.358 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:24.358 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:24.358 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:24.358 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:24.359 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:24.359 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:24.359 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:24.359 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:24.359 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:24.359 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:24.359 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:24.360 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:24.360 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:24.360 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:24.360 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:24.360 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:24.360 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:24.360 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:24.360 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:24.361 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:24.361 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:24.361 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:24.361 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:24.361 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:24.361 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:24.365 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:24.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:24.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:24.467 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:24.468 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:24.468 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:24.468 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:24.468 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:24.469 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:24.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:17:24.863 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:24.864 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:24.864 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:25.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 3 2020-09-12T14:17:25.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:17:25.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:25.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:25.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:25.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:25.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:25.643 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:25.643 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:26.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=9173309 2020-09-12T14:17:26.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 9173309 '!=' null 2020-09-12T14:17:26.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:26.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:26.233 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:26.233 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:26.335 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:26.335 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:26.335 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:26.335 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:26.335 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:26.336 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:26.336 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:26.336 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:26.336 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:26.336 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:26.336 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:26.336 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:26.337 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:26.337 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:26.337 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:26.337 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:26.337 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:26.337 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:26.337 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:26.337 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:26.338 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:26.338 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:26.338 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:26.338 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:26.338 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:26.338 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:26.338 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:26.339 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:26.339 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:26.339 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:26.339 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:26.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:26.444 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:26.445 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:26.445 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:26.445 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:26.446 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:26.446 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:26.446 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:26.446 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:26.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:17:26.849 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:26.850 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:26.850 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:27.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 3 2020-09-12T14:17:27.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:17:27.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:27.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:27.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:27.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:27.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:27.394 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:27.394 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:28.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=19657365 2020-09-12T14:17:28.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 19657365 '!=' null 2020-09-12T14:17:28.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:28.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:28.021 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:28.021 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:28.122 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:28.123 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:28.123 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:28.123 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:28.123 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:28.123 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:28.123 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:28.123 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:28.124 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:28.124 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:28.124 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:28.124 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:28.124 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:28.124 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:28.124 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:28.124 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:28.125 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:28.125 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:28.125 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:28.125 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:28.125 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:28.125 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:28.125 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:28.125 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:28.126 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:28.126 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:28.126 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:28.126 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:28.126 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:28.126 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:28.126 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:28.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:28.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:28.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:28.233 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:28.233 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:28.233 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:28.234 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:28.234 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:28.234 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:29.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:17:29.016 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:29.017 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:29.017 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:29.548 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 3 2020-09-12T14:17:29.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 1 '!=' 0 2020-09-12T14:17:29.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1598: wait_for_clean: loop=0 2020-09-12T14:17:29.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1599: wait_for_clean: num_active_clean=1 2020-09-12T14:17:29.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:29.550 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:29.550 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:29.650 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:29.651 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:29.651 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:29.651 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:29.652 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:29.652 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:29.652 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:29.652 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:29.653 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:29.653 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:29.653 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:29.654 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:29.654 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:29.654 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:29.654 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:29.655 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:29.655 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:29.655 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:29.656 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:29.656 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:29.656 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:29.656 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:29.657 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:29.657 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:29.657 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:29.658 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:29.658 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:29.658 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:29.659 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:29.659 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:29.659 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:29.660 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:29.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:29.761 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:29.761 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:29.762 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:29.762 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:29.762 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:29.762 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:29.762 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:30.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:17:30.159 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:30.160 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:30.160 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:30.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 3 2020-09-12T14:17:30.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 1 '!=' 1 2020-09-12T14:17:30.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:30.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:30.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:30.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:30.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:30.708 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:30.709 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:31.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=17162327 2020-09-12T14:17:31.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 17162327 '!=' null 2020-09-12T14:17:31.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:31.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:31.314 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:31.314 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:31.415 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:31.415 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:31.415 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:31.415 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:31.416 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:31.416 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:31.416 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:31.416 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:31.416 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:31.416 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:31.416 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:31.416 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:31.417 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:31.417 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:31.417 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:31.417 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:31.417 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:31.417 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:31.417 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:31.418 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:31.418 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:31.418 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:31.418 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:31.418 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:31.418 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:31.418 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:31.418 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:31.419 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:31.419 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:31.419 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:31.419 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:31.423 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:31.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:31.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:31.525 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:31.525 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:31.525 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:31.526 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:31.526 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:31.526 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:32.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:17:32.101 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:32.101 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:32.101 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:32.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 3 2020-09-12T14:17:32.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 1 '!=' 1 2020-09-12T14:17:32.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:32.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:32.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:32.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:32.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:32.643 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:32.644 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:33.347 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=17399467 2020-09-12T14:17:33.347 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 17399467 '!=' null 2020-09-12T14:17:33.347 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:33.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:33.348 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:33.348 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:33.451 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:33.452 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:33.452 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:33.452 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:33.452 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:33.452 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:33.452 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:33.453 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:33.453 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:33.453 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:33.453 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:33.453 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:33.453 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:33.453 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:33.454 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:33.454 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:33.454 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:33.454 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:33.454 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:33.454 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:33.454 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:33.454 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:33.455 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:33.455 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:33.455 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:33.455 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:33.455 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:33.455 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:33.455 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:33.455 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:33.456 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:33.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:33.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:33.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:33.562 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:33.562 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:33.562 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:33.563 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:33.563 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:33.563 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:33.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:17:33.959 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:33.959 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:33.959 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:34.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 3 2020-09-12T14:17:34.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 1 '!=' 1 2020-09-12T14:17:34.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:34.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:34.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:34.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:34.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:34.780 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:34.780 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:35.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=14852117 2020-09-12T14:17:35.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 14852117 '!=' null 2020-09-12T14:17:35.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:35.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:35.325 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:35.325 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:35.429 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:35.429 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:35.430 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:35.430 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:35.430 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:35.431 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:35.431 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:35.431 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:35.431 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:35.431 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:35.432 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:35.432 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:35.432 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:35.432 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:35.432 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:35.432 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:35.433 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:35.433 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:35.433 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:35.433 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:35.433 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:35.433 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:35.433 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:35.434 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:35.434 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:35.434 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:35.434 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:35.434 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:35.434 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:35.434 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:35.435 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:35.437 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:35.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:35.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:35.540 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:35.540 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:35.540 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:35.540 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:35.540 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:35.540 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:35.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:17:35.947 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:35.947 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:35.948 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:36.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 3 2020-09-12T14:17:36.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 1 '!=' 1 2020-09-12T14:17:36.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:36.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:36.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:36.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:36.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:36.494 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:36.494 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:37.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=14852038 2020-09-12T14:17:37.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 14852038 '!=' null 2020-09-12T14:17:37.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:37.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:37.219 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:37.219 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:37.320 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:37.320 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:37.321 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:37.321 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:37.321 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:37.321 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:37.321 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:37.321 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:37.321 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:37.321 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:37.322 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:37.322 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:37.322 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:37.322 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:37.322 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:37.322 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:37.323 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:37.323 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:37.323 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:37.323 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:37.323 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:37.323 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:37.323 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:37.324 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:37.324 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:37.324 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:37.324 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:37.324 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:37.324 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:37.325 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:37.325 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:37.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:37.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:37.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:37.433 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:37.433 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:37.433 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:37.433 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:37.433 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:37.434 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:38.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:17:38.037 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:38.037 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:38.037 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:38.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 3 2020-09-12T14:17:38.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 1 '!=' 1 2020-09-12T14:17:38.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:38.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:38.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:38.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:38.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:38.579 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:38.579 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:39.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=20967686 2020-09-12T14:17:39.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 20967686 '!=' null 2020-09-12T14:17:39.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:39.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:39.171 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:39.171 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:39.272 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:39.272 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:39.272 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:39.273 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:39.273 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:39.273 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:39.273 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:39.273 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:39.273 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:39.273 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:39.273 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:39.274 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:39.274 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:39.274 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:39.274 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:39.274 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:39.274 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:39.274 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:39.275 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:39.275 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:39.275 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:39.275 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:39.275 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:39.275 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:39.275 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:39.275 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:39.276 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:39.276 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:39.276 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:39.276 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:39.276 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:39.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:39.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:39.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:39.383 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:39.383 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:39.383 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:39.383 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:39.383 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:39.383 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:39.785 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:17:39.786 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:39.786 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:39.787 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:40.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 3 2020-09-12T14:17:40.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 1 '!=' 1 2020-09-12T14:17:40.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:40.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:40.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:40.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:40.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:40.327 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:40.327 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:41.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=13978298 2020-09-12T14:17:41.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 13978298 '!=' null 2020-09-12T14:17:41.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:41.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:41.109 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:41.110 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:41.215 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:41.215 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:41.216 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:41.216 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:41.216 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:41.216 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:41.217 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:41.217 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:41.217 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:41.217 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:41.217 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:41.218 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:41.218 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:41.218 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:41.218 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:41.219 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:41.219 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:41.219 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:41.219 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:41.220 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:41.220 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:41.220 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:41.220 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:41.221 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:41.221 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:41.221 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:41.221 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:41.221 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:41.222 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:41.222 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:41.222 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:41.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:41.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:41.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:41.325 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:41.325 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:41.326 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:41.326 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:41.326 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:41.326 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:41.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:17:41.726 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:41.726 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:41.726 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:42.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 3 2020-09-12T14:17:42.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 1 '!=' 1 2020-09-12T14:17:42.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:42.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:42.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:42.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:42.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:42.267 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:42.267 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:42.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=20967480 2020-09-12T14:17:42.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 20967480 '!=' null 2020-09-12T14:17:42.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:42.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:42.889 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:42.890 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:42.993 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:42.993 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:42.993 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:42.993 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:42.993 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:42.994 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:42.994 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:42.994 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:42.994 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:42.994 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:42.994 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:42.994 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:42.995 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:42.995 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:42.995 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:42.995 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:42.995 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:42.995 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:42.995 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:42.995 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:42.996 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:42.996 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:42.996 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:42.996 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:42.996 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:42.996 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:42.996 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:42.996 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:42.997 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:42.997 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:42.997 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:43.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:43.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:43.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:43.104 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:43.104 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:43.104 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:43.104 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:43.104 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:43.105 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:43.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:17:43.507 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:43.507 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:43.507 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:44.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 3 2020-09-12T14:17:44.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 1 '!=' 1 2020-09-12T14:17:44.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:44.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:44.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:44.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:44.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:44.280 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:44.280 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:44.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=13104569 2020-09-12T14:17:44.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 13104569 '!=' null 2020-09-12T14:17:44.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:44.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:44.835 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:44.835 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:44.937 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:44.937 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:44.937 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:44.937 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:44.938 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:44.938 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:44.938 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:44.938 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:44.938 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:44.938 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:44.939 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:44.939 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:44.939 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:44.939 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:44.939 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:44.939 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:44.939 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:44.940 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:44.940 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:44.940 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:44.940 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:44.940 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:44.940 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:44.940 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:44.940 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:44.941 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:44.941 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:44.941 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:44.941 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:44.941 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:44.941 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:44.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:45.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:45.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:45.047 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:45.047 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:45.048 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:45.048 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:45.048 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:45.048 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:45.444 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:17:45.445 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:45.445 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:45.446 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:46.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 3 2020-09-12T14:17:46.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 1 '!=' 1 2020-09-12T14:17:46.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:46.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:46.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:46.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:46.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:46.037 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:46.037 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:46.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=13104665 2020-09-12T14:17:46.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 13104665 '!=' null 2020-09-12T14:17:46.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:46.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:46.876 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:46.876 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:46.985 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:46.985 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:46.985 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:46.986 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:46.986 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:46.986 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:46.986 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:46.986 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:46.986 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:46.986 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:46.987 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:46.987 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:46.987 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:46.987 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:46.987 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:46.987 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:46.987 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:46.988 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:46.988 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:46.988 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:46.988 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:46.988 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:46.988 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:46.988 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:46.988 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:46.989 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:46.989 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:46.989 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:46.989 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:46.989 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:46.989 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:46.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:47.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:47.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:47.096 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:47.096 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:47.096 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:47.096 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:47.097 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:47.097 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:47.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:17:47.494 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:47.494 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:47.495 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:48.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 3 2020-09-12T14:17:48.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 1 '!=' 1 2020-09-12T14:17:48.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:48.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:48.038 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:48.038 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:48.038 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:48.038 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:48.038 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:48.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=20094003 2020-09-12T14:17:48.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 20094003 '!=' null 2020-09-12T14:17:48.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:48.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:48.603 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:48.604 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:48.705 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:48.706 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:48.706 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:48.706 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:48.706 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:48.706 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:48.706 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:48.707 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:48.707 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:48.707 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:48.707 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:48.707 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:48.707 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:48.707 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:48.707 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:48.708 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:48.708 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:48.708 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:48.708 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:48.708 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:48.708 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:48.708 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:48.708 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:48.709 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:48.709 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:48.709 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:48.709 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:48.709 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:48.709 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:48.709 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:48.709 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:48.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:48.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:48.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:48.816 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:48.816 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:48.816 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:48.816 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:48.816 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:48.816 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:49.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:17:49.221 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:49.222 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:49.222 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:50.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 3 2020-09-12T14:17:50.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 1 '!=' 1 2020-09-12T14:17:50.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:50.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:50.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:50.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:50.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:50.128 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:50.128 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:50.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=13978436 2020-09-12T14:17:50.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 13978436 '!=' null 2020-09-12T14:17:50.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:50.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:50.708 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:50.708 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:50.810 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:50.810 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:50.810 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:50.810 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:50.810 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [ 2020-09-12T14:17:50.811 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:50.811 INFO:tasks.workunit.client.0.smithi027.stdout: "priority": 110, 2020-09-12T14:17:50.811 INFO:tasks.workunit.client.0.smithi027.stdout: "items": [ 2020-09-12T14:17:50.811 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:50.811 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:50.811 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:50.811 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:50.812 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:50.812 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:50.812 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:50.812 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:17:50.812 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:50.812 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:50.812 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "3.0", 2020-09-12T14:17:50.812 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 151, 2020-09-12T14:17:50.813 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:50.813 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:50.813 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:50.813 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:50.813 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:50.813 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:50.813 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:50.813 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:50.814 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:50.814 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:50.814 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:50.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:50.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:50.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:50.920 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:50.920 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:50.921 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:50.921 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:50.921 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:50.921 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:51.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:17:51.322 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:51.323 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:51.323 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:51.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 3 2020-09-12T14:17:51.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 1 '!=' 1 2020-09-12T14:17:51.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:51.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:51.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:51.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:51.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:51.856 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:51.856 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:52.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=20967685 2020-09-12T14:17:52.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 20967685 '!=' null 2020-09-12T14:17:52.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:52.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:52.394 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:52.394 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:52.496 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:52.496 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:52.497 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:52.497 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:52.497 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:52.497 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:52.497 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:52.497 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:52.497 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:52.498 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:52.498 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:52.498 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:52.498 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:52.498 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:52.498 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:52.498 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:52.499 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:52.499 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:52.499 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:52.499 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:52.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:52.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:52.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:52.609 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:52.610 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:52.610 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:52.610 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:52.610 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:52.610 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:53.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:17:53.182 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:53.182 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:53.183 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:53.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 3 2020-09-12T14:17:53.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 1 '!=' 1 2020-09-12T14:17:53.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:53.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:53.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:53.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:53.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:53.830 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:53.831 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:54.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=16774308 2020-09-12T14:17:54.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 16774308 '!=' null 2020-09-12T14:17:54.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:54.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:54.409 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:54.409 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:54.511 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:54.512 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:54.512 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:54.512 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:54.512 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:54.512 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:54.513 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:54.513 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:54.513 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:54.513 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:54.513 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:54.513 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:54.513 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:54.513 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:54.514 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:54.514 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:54.514 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:54.514 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:54.514 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:54.514 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:54.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:54.621 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:54.621 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:54.622 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:54.622 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:54.622 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:54.622 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:54.623 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:54.623 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:55.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:17:55.105 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:55.105 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:55.105 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:55.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 3 2020-09-12T14:17:55.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 1 '!=' 1 2020-09-12T14:17:55.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:55.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:55.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:55.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:55.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:55.657 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:55.657 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:56.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=10483849 2020-09-12T14:17:56.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 10483849 '!=' null 2020-09-12T14:17:56.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:56.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:56.288 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:56.289 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:56.391 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:56.391 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:56.391 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:56.391 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:56.391 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:56.392 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:56.392 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:56.392 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:56.392 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:56.392 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:56.392 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:56.392 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:56.392 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:56.393 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:56.393 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:56.393 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:56.393 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:56.393 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:56.393 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:56.393 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:56.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:56.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:56.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:56.501 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:56.501 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:56.501 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:56.502 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:56.502 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:56.502 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:56.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:17:56.903 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:56.903 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:56.903 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:57.445 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 3 2020-09-12T14:17:57.445 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 1 '!=' 1 2020-09-12T14:17:57.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:17:57.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:17:57.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:17:57.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:17:57.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:17:57.446 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:17:57.447 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:17:58.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=18346744 2020-09-12T14:17:58.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 18346744 '!=' null 2020-09-12T14:17:58.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:17:58.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:58.001 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:58.001 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:58.103 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:58.103 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:58.104 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:58.104 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:58.104 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:58.104 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:58.104 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:58.104 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:58.104 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:58.105 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:58.105 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:58.105 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:58.105 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:58.105 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:58.105 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:58.105 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:58.105 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:58.106 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:58.106 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:58.106 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:58.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:58.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:58.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:58.214 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:58.214 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:58.214 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:58.214 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:58.215 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:58.215 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:58.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=2 2020-09-12T14:17:58.616 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:58.616 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:58.616 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:17:59.284 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 2 = 3 2020-09-12T14:17:59.284 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 2 '!=' 1 2020-09-12T14:17:59.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1598: wait_for_clean: loop=0 2020-09-12T14:17:59.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1599: wait_for_clean: num_active_clean=2 2020-09-12T14:17:59.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:59.285 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:17:59.285 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:17:59.386 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:17:59.387 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:17:59.387 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:59.387 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:59.388 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:59.388 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:17:59.388 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:17:59.388 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:17:59.388 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:17:59.388 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:17:59.388 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:59.388 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:17:59.389 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:17:59.389 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:17:59.389 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:17:59.389 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:17:59.389 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:17:59.389 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:17:59.389 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:17:59.390 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:17:59.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:17:59.496 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:17:59.496 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:17:59.497 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:17:59.497 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:17:59.497 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:17:59.497 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:17:59.497 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:17:59.498 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:17:59.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=2 2020-09-12T14:17:59.907 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:17:59.908 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:17:59.909 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:18:00.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 2 = 3 2020-09-12T14:18:00.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 2 '!=' 2 2020-09-12T14:18:00.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:18:00.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:18:00.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:18:00.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:18:00.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:18:00.443 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:18:00.443 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:18:01.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=7863014 2020-09-12T14:18:01.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 7863014 '!=' null 2020-09-12T14:18:01.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:18:01.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:01.053 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:18:01.053 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:01.155 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:18:01.155 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:18:01.156 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:01.156 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:01.156 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:01.156 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:18:01.156 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:01.156 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:18:01.156 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:18:01.157 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:18:01.157 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:01.157 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:01.157 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:01.157 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:18:01.157 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:01.157 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:01.158 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:01.158 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:18:01.158 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:01.158 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:18:01.164 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:18:01.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:18:01.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:18:01.266 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:18:01.266 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:18:01.266 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:18:01.266 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:18:01.267 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:18:01.267 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:18:01.661 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=2 2020-09-12T14:18:01.662 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:18:01.662 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:18:01.662 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:18:02.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 2 = 3 2020-09-12T14:18:02.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 2 '!=' 2 2020-09-12T14:18:02.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:18:02.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:18:02.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:18:02.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:18:02.236 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:18:02.236 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:18:02.236 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:18:02.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=17279045 2020-09-12T14:18:02.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 17279045 '!=' null 2020-09-12T14:18:02.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:18:02.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:02.774 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:18:02.774 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:02.876 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:18:02.876 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:18:02.876 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:02.876 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:02.876 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:02.877 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:18:02.877 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:02.877 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:18:02.877 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:18:02.877 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:18:02.877 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:02.877 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:02.878 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:02.878 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:18:02.878 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:02.878 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:02.878 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:02.878 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:18:02.878 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:02.878 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:18:02.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:18:02.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:18:02.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:18:02.987 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:18:02.987 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:18:02.987 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:18:02.987 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:18:02.987 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:18:02.987 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:18:03.470 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=2 2020-09-12T14:18:03.470 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:18:03.471 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:18:03.471 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:18:04.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 2 = 3 2020-09-12T14:18:04.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 2 '!=' 2 2020-09-12T14:18:04.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:18:04.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:18:04.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:18:04.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:18:04.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:18:04.023 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:18:04.023 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:18:04.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=14677629 2020-09-12T14:18:04.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 14677629 '!=' null 2020-09-12T14:18:04.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:18:04.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:04.570 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:18:04.570 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:04.676 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:18:04.677 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:18:04.677 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:04.677 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:04.677 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:04.677 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:18:04.677 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:04.677 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:18:04.678 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:18:04.678 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:18:04.678 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:04.678 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:04.678 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:04.678 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:18:04.678 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:04.679 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:04.679 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:04.679 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:18:04.679 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:04.679 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:18:04.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:18:04.785 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:18:04.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:18:04.787 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:18:04.787 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:18:04.787 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:18:04.787 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:18:04.787 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:18:04.787 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:18:05.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=2 2020-09-12T14:18:05.179 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:18:05.180 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:18:05.180 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:18:05.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 2 = 3 2020-09-12T14:18:05.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 2 '!=' 2 2020-09-12T14:18:05.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:18:05.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:18:05.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:18:05.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:18:05.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:18:05.804 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:18:05.805 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:18:06.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=12313685 2020-09-12T14:18:06.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 12313685 '!=' null 2020-09-12T14:18:06.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:18:06.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:06.421 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:18:06.421 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:06.524 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:18:06.524 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:18:06.524 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:06.525 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:06.525 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:06.525 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:18:06.525 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:06.525 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:18:06.525 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:18:06.525 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:18:06.526 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:06.526 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:06.526 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:06.526 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:18:06.526 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:06.526 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:06.526 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:06.527 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:18:06.527 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:06.527 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:18:06.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:18:06.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:18:06.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:18:06.634 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:18:06.635 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:18:06.635 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:18:06.635 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:18:06.635 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:18:06.635 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:18:07.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=2 2020-09-12T14:18:07.034 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:18:07.035 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:18:07.035 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:18:07.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 2 = 3 2020-09-12T14:18:07.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 2 '!=' 2 2020-09-12T14:18:07.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:18:07.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:18:07.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:18:07.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:18:07.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:18:07.575 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:18:07.575 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:18:08.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=19220751 2020-09-12T14:18:08.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 19220751 '!=' null 2020-09-12T14:18:08.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:18:08.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:08.475 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:18:08.475 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:08.580 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:18:08.580 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:18:08.580 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:08.580 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:08.580 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:08.581 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:18:08.581 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:08.581 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:18:08.581 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:18:08.581 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:18:08.581 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:08.581 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:08.582 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:08.582 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:18:08.582 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:08.582 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:08.582 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:08.582 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:18:08.582 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:08.583 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:18:08.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:18:08.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:18:08.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:18:08.690 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:18:08.690 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:18:08.690 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:18:08.690 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:18:08.691 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:18:08.691 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:18:09.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=2 2020-09-12T14:18:09.088 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:18:09.089 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:18:09.089 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:18:09.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 2 = 3 2020-09-12T14:18:09.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 2 '!=' 2 2020-09-12T14:18:09.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:18:09.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:18:09.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:18:09.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:18:09.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:18:09.756 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:18:09.757 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:18:10.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=13978744 2020-09-12T14:18:10.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 13978744 '!=' null 2020-09-12T14:18:10.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:18:10.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:10.302 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:18:10.302 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:10.405 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:18:10.405 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:18:10.405 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:10.405 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:10.405 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:10.406 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:18:10.406 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:10.406 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:18:10.406 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:18:10.406 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:18:10.406 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:10.406 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:10.407 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:10.407 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:18:10.407 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:10.407 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:10.407 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:10.407 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:18:10.407 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:10.408 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:18:10.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:18:10.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:18:10.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:18:10.515 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:18:10.515 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:18:10.516 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:18:10.516 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:18:10.516 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:18:10.516 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:18:10.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=2 2020-09-12T14:18:10.913 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:18:10.913 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:18:10.913 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:18:11.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 2 = 3 2020-09-12T14:18:11.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 2 '!=' 2 2020-09-12T14:18:11.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:18:11.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:18:11.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:18:11.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:18:11.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:18:11.459 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:18:11.460 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:18:12.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=20094471 2020-09-12T14:18:12.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 20094471 '!=' null 2020-09-12T14:18:12.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:18:12.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:12.158 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:18:12.159 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:12.260 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:18:12.260 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:18:12.260 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:12.261 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:12.261 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:12.261 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:18:12.261 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:12.262 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:18:12.262 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:18:12.262 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:18:12.263 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:12.263 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:12.263 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:12.263 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:18:12.264 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:12.264 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:12.264 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:12.264 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:18:12.265 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:12.265 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:18:12.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:18:12.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:18:12.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:18:12.370 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:18:12.370 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:18:12.370 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:18:12.370 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:18:12.371 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:18:12.371 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:18:12.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=2 2020-09-12T14:18:12.764 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:18:12.765 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:18:12.765 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:18:13.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 2 = 3 2020-09-12T14:18:13.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 2 '!=' 2 2020-09-12T14:18:13.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:18:13.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:18:13.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:18:13.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:18:13.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:18:13.375 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:18:13.376 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:18:14.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=13105057 2020-09-12T14:18:14.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 13105057 '!=' null 2020-09-12T14:18:14.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:18:14.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:14.228 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:18:14.228 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:14.334 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:18:14.334 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:18:14.334 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:14.334 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:14.335 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:14.335 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:18:14.335 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:14.335 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:18:14.335 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:18:14.335 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:18:14.335 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:14.336 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:14.336 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:14.336 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:18:14.336 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:14.336 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:14.336 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:14.336 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:18:14.337 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:14.337 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:18:14.342 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:18:14.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:18:14.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:18:14.444 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:18:14.444 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:18:14.445 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:18:14.445 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:18:14.445 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:18:14.445 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:18:14.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=2 2020-09-12T14:18:14.846 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:18:14.847 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:18:14.847 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:18:15.378 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 2 = 3 2020-09-12T14:18:15.378 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 2 '!=' 2 2020-09-12T14:18:15.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:18:15.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:18:15.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:18:15.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:18:15.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:18:15.379 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:18:15.379 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:18:16.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=13105024 2020-09-12T14:18:16.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 13105024 '!=' null 2020-09-12T14:18:16.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:18:16.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:16.078 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:18:16.078 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:16.180 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:18:16.181 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:18:16.181 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:16.181 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:16.181 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:16.181 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:18:16.182 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:16.182 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:18:16.182 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:18:16.182 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:18:16.182 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:16.182 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:16.182 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:16.182 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:18:16.183 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:16.183 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:16.183 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:16.183 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:18:16.183 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:16.183 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:18:16.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:18:16.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:18:16.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:18:16.291 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:18:16.291 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:18:16.291 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:18:16.291 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:18:16.292 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:18:16.292 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:18:16.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=2 2020-09-12T14:18:16.693 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:18:16.694 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:18:16.694 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:18:17.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 2 = 3 2020-09-12T14:18:17.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 2 '!=' 2 2020-09-12T14:18:17.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:18:17.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:18:17.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:18:17.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:18:17.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:18:17.520 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:18:17.520 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:18:18.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=20968045 2020-09-12T14:18:18.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 20968045 '!=' null 2020-09-12T14:18:18.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:18:18.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:18.053 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:18:18.053 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:18.156 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:18:18.156 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:18:18.156 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:18.156 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:18.157 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:18.157 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:18:18.157 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:18.157 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:18:18.157 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:18:18.157 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:18:18.157 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:18.158 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:18.158 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:18.158 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:18:18.158 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:18.158 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:18.158 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:18.159 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:18:18.159 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:18.159 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:18:18.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:18:18.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:18:18.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:18:18.269 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:18:18.269 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:18:18.269 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:18:18.269 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:18:18.269 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:18:18.269 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:18:18.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=2 2020-09-12T14:18:18.736 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:18:18.736 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:18:18.737 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:18:19.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 2 = 3 2020-09-12T14:18:19.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 2 '!=' 2 2020-09-12T14:18:19.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:18:19.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:18:19.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:18:19.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:18:19.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:18:19.278 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:18:19.279 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:18:19.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=13978714 2020-09-12T14:18:19.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 13978714 '!=' null 2020-09-12T14:18:19.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:18:19.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:19.818 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:18:19.818 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:19.920 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:18:19.920 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:18:19.920 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:19.921 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:19.921 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:19.921 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:18:19.921 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:19.921 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:18:19.921 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:18:19.921 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:18:19.922 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:19.922 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:19.922 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:19.922 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:18:19.922 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:19.922 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:19.922 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:19.922 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:18:19.923 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:19.923 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:18:19.928 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:18:20.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:18:20.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:18:20.030 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:18:20.030 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:18:20.030 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:18:20.030 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:18:20.031 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:18:20.031 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:18:20.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=2 2020-09-12T14:18:20.549 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:18:20.549 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:18:20.550 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:18:21.085 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 2 = 3 2020-09-12T14:18:21.085 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 2 '!=' 2 2020-09-12T14:18:21.085 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:18:21.085 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:18:21.086 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:18:21.086 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:18:21.086 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:18:21.086 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:18:21.086 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:18:21.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=20968066 2020-09-12T14:18:21.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 20968066 '!=' null 2020-09-12T14:18:21.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:18:21.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:21.726 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:18:21.726 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:21.828 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:18:21.828 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:18:21.829 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:21.829 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:21.829 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:21.829 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:18:21.829 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:21.829 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:18:21.829 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:18:21.830 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:18:21.830 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:21.830 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:21.830 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:21.830 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:18:21.830 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:21.830 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:21.831 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:21.831 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:18:21.831 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:21.831 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:18:21.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:18:21.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:18:21.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:18:21.939 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:18:21.939 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:18:21.939 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:18:21.939 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:18:21.940 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:18:21.940 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:18:22.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=2 2020-09-12T14:18:22.347 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:18:22.347 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:18:22.347 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:18:22.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 2 = 3 2020-09-12T14:18:22.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 2 '!=' 2 2020-09-12T14:18:22.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:18:22.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:18:22.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:18:22.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:18:22.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:18:22.912 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:18:22.912 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:18:23.482 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=20968066 2020-09-12T14:18:23.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 20968066 '!=' null 2020-09-12T14:18:23.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:18:23.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:23.483 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:18:23.483 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:23.585 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:18:23.586 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:18:23.586 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:23.586 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:23.586 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:23.586 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:18:23.586 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:23.587 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:18:23.587 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:18:23.587 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:18:23.587 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:23.587 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:23.587 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:23.587 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:18:23.587 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:23.588 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:23.588 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:23.588 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:18:23.588 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:23.588 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:18:23.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:18:23.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:18:23.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:18:23.696 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:18:23.696 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:18:23.697 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:18:23.697 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:18:23.697 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:18:23.697 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:18:24.191 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=2 2020-09-12T14:18:24.191 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:18:24.192 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:18:24.192 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:18:24.728 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 2 = 3 2020-09-12T14:18:24.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 2 '!=' 2 2020-09-12T14:18:24.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:18:24.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:18:24.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:18:24.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:18:24.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:18:24.730 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:18:24.730 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:18:25.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=14852387 2020-09-12T14:18:25.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 14852387 '!=' null 2020-09-12T14:18:25.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:18:25.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:25.576 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:18:25.576 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:25.678 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:18:25.678 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:18:25.679 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:25.679 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:25.679 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:25.679 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [ 2020-09-12T14:18:25.679 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:25.679 INFO:tasks.workunit.client.0.smithi027.stdout: "item": "1.0", 2020-09-12T14:18:25.679 INFO:tasks.workunit.client.0.smithi027.stdout: "prio": 110, 2020-09-12T14:18:25.680 INFO:tasks.workunit.client.0.smithi027.stdout: "can_preempt": true 2020-09-12T14:18:25.680 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:25.680 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:25.680 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:25.680 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:18:25.680 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:25.680 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:25.680 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:25.681 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:18:25.681 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:25.681 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:18:25.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:18:25.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:18:25.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:18:25.790 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:18:25.790 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:18:25.791 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:18:25.791 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:18:25.791 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:18:25.791 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:18:26.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=2 2020-09-12T14:18:26.194 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:18:26.194 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:18:26.194 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:18:26.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 2 = 3 2020-09-12T14:18:26.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 2 '!=' 2 2020-09-12T14:18:26.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:18:26.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:18:26.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:18:26.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:18:26.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:18:26.840 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:18:26.840 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:18:27.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=14852345 2020-09-12T14:18:27.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 14852345 '!=' null 2020-09-12T14:18:27.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:18:27.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:27.382 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:18:27.382 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:27.485 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:18:27.485 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:18:27.485 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:27.485 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:27.486 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:27.486 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:18:27.486 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:27.486 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:18:27.486 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:27.486 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:27.486 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:27.486 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:18:27.487 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:27.487 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:18:27.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:18:27.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:18:27.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:18:27.595 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:18:27.595 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:18:27.595 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:18:27.596 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:18:27.596 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:18:27.596 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:18:27.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=2 2020-09-12T14:18:27.994 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:18:27.995 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:18:27.995 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:18:28.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 2 = 3 2020-09-12T14:18:28.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 2 '!=' 2 2020-09-12T14:18:28.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:18:28.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:18:28.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:18:28.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:18:28.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:18:28.540 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:18:28.541 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:18:29.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=18347080 2020-09-12T14:18:29.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 18347080 '!=' null 2020-09-12T14:18:29.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:18:29.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:29.141 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:18:29.141 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:29.243 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:18:29.243 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:18:29.244 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:29.244 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:29.244 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:29.244 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:18:29.244 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:29.244 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:18:29.244 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:29.244 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:29.245 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:29.245 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:18:29.245 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:29.245 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:18:29.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:18:29.353 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:18:29.353 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:18:29.353 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:18:29.353 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:18:29.354 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:18:29.354 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:18:29.354 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:18:29.354 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:18:29.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=2 2020-09-12T14:18:29.753 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:18:29.753 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:18:29.753 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:18:30.284 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 2 = 3 2020-09-12T14:18:30.284 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 2 '!=' 2 2020-09-12T14:18:30.284 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:18:30.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:18:30.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:18:30.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:18:30.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:18:30.285 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:18:30.285 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:18:30.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=5241998 2020-09-12T14:18:30.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 5241998 '!=' null 2020-09-12T14:18:30.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:18:30.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:30.825 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:18:30.825 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:30.927 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:18:30.927 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:18:30.927 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:30.928 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:30.928 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:30.928 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:18:30.928 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:30.928 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:18:30.928 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:30.928 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:30.929 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:30.929 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:18:30.929 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:30.929 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:18:30.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:18:31.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:18:31.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:18:31.037 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:18:31.038 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:18:31.038 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:18:31.038 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:18:31.038 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:18:31.038 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:18:31.437 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=2 2020-09-12T14:18:31.438 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:18:31.438 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:18:31.438 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:18:31.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 2 = 3 2020-09-12T14:18:31.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 2 '!=' 2 2020-09-12T14:18:31.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:18:31.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:18:31.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:18:31.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:18:31.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:18:31.975 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:18:31.975 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:18:32.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=7862986 2020-09-12T14:18:32.510 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 7862986 '!=' null 2020-09-12T14:18:32.510 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:18:32.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 'CEPH_ARGS='\'''\''' ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:32.511 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: CEPH_ARGS= 2020-09-12T14:18:32.511 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_recovery_reservations 2020-09-12T14:18:32.612 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:18:32.612 INFO:tasks.workunit.client.0.smithi027.stdout: "local_reservations": { 2020-09-12T14:18:32.613 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:32.613 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:32.613 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:32.613 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:18:32.614 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:32.614 INFO:tasks.workunit.client.0.smithi027.stdout: "remote_reservations": { 2020-09-12T14:18:32.614 INFO:tasks.workunit.client.0.smithi027.stdout: "max_allowed": 1, 2020-09-12T14:18:32.614 INFO:tasks.workunit.client.0.smithi027.stdout: "min_priority": 0, 2020-09-12T14:18:32.615 INFO:tasks.workunit.client.0.smithi027.stdout: "queues": [], 2020-09-12T14:18:32.615 INFO:tasks.workunit.client.0.smithi027.stdout: "in_progress": [] 2020-09-12T14:18:32.615 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:32.616 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:18:32.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:18:32.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:18:32.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:18:32.722 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:18:32.722 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:18:32.722 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:18:32.723 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:18:32.723 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:18:32.723 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:18:33.122 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=3 2020-09-12T14:18:33.122 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:18:33.123 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:18:33.123 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:18:33.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 3 = 3 2020-09-12T14:18:33.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T14:18:33.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T14:18:33.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:298: TEST_backfill_priority: ceph pg dump pgs 2020-09-12T14:18:34.036 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:18:34.037 INFO:tasks.workunit.client.0.smithi027.stdout:7.0 50 0 0 0 0 524288000 0 0 50 50 active+clean 2020-09-12T14:17:21.585510+0000 93'150 119:343 [1,4] 1 [1,4] 1 0'0 2020-09-12T14:13:46.480650+0000 0'0 2020-09-12T14:13:46.480650+0000 0 2020-09-12T14:18:34.037 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 50 0 0 0 0 524288000 0 0 50 50 active+clean 2020-09-12T14:18:28.275516+0000 93'150 120:350 [1,2] 1 [1,2] 1 0'0 2020-09-12T14:13:30.255666+0000 0'0 2020-09-12T14:13:30.255666+0000 0 2020-09-12T14:18:34.037 INFO:tasks.workunit.client.0.smithi027.stdout:3.0 50 0 0 0 0 524288000 0 0 50 50 active+clean 2020-09-12T14:17:53.886542+0000 93'150 119:358 [1,2] 1 [1,2] 1 0'0 2020-09-12T14:13:35.363167+0000 0'0 2020-09-12T14:13:35.363167+0000 0 2020-09-12T14:18:34.037 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:18:34.037 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:18:34.038 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:18:34.048 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:300: TEST_backfill_priority: get_asok_path osd.1 2020-09-12T14:18:34.048 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2020-09-12T14:18:34.049 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2020-09-12T14:18:34.049 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:18:34.049 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:18:34.049 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:18:34.049 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.73258/ceph-osd.1.asok 2020-09-12T14:18:34.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:300: TEST_backfill_priority: CEPH_ARGS= 2020-09-12T14:18:34.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:300: TEST_backfill_priority: ceph --admin-daemon /tmp/ceph-asok.73258/ceph-osd.1.asok dump_pgstate_history 2020-09-12T14:18:34.153 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:18:34.153 INFO:tasks.workunit.client.0.smithi027.stdout: "pgs": [ 2020-09-12T14:18:34.153 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.153 INFO:tasks.workunit.client.0.smithi027.stdout: "pg": "7.0", 2020-09-12T14:18:34.154 INFO:tasks.workunit.client.0.smithi027.stdout: "currently": "Started/Primary/Active/Clean", 2020-09-12T14:18:34.154 INFO:tasks.workunit.client.0.smithi027.stdout: "history": [ 2020-09-12T14:18:34.154 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.154 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "64", 2020-09-12T14:18:34.154 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.154 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.155 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Start", 2020-09-12T14:18:34.155 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:47.228160+0000", 2020-09-12T14:18:34.155 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:47.228181+0000" 2020-09-12T14:18:34.155 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.155 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.155 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetInfo", 2020-09-12T14:18:34.155 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:47.228193+0000", 2020-09-12T14:18:34.156 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:47.383707+0000" 2020-09-12T14:18:34.156 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.156 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.156 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetLog", 2020-09-12T14:18:34.156 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:47.383708+0000", 2020-09-12T14:18:34.156 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:47.383772+0000" 2020-09-12T14:18:34.156 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.156 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.157 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetMissing", 2020-09-12T14:18:34.157 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:47.383772+0000", 2020-09-12T14:18:34.157 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:47.383784+0000" 2020-09-12T14:18:34.157 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.157 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.157 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/WaitUpThru", 2020-09-12T14:18:34.157 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:47.383784+0000", 2020-09-12T14:18:34.157 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:47.512879+0000" 2020-09-12T14:18:34.158 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.158 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.158 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering", 2020-09-12T14:18:34.158 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:47.228187+0000", 2020-09-12T14:18:34.158 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:47.512887+0000" 2020-09-12T14:18:34.158 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.158 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.159 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Activating", 2020-09-12T14:18:34.159 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:47.513122+0000", 2020-09-12T14:18:34.159 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:48.115708+0000" 2020-09-12T14:18:34.159 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.159 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.159 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Recovered", 2020-09-12T14:18:34.159 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:48.115709+0000", 2020-09-12T14:18:34.159 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:48.115819+0000" 2020-09-12T14:18:34.160 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.160 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.160 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Clean", 2020-09-12T14:18:34.160 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:48.115819+0000", 2020-09-12T14:18:34.160 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:48.441789+0000" 2020-09-12T14:18:34.160 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.160 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.161 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active", 2020-09-12T14:18:34.161 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:47.512887+0000", 2020-09-12T14:18:34.161 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:48.441809+0000" 2020-09-12T14:18:34.161 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.161 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.161 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary", 2020-09-12T14:18:34.161 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:47.228181+0000", 2020-09-12T14:18:34.162 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:48.441823+0000" 2020-09-12T14:18:34.162 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.162 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.162 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started", 2020-09-12T14:18:34.162 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:47.228155+0000", 2020-09-12T14:18:34.162 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:48.441826+0000" 2020-09-12T14:18:34.162 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.162 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.163 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.163 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.163 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "64", 2020-09-12T14:18:34.163 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.163 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.163 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Reset", 2020-09-12T14:18:34.163 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:48.441827+0000", 2020-09-12T14:18:34.163 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:48.442043+0000" 2020-09-12T14:18:34.164 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.164 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.164 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.164 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.164 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "91", 2020-09-12T14:18:34.164 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.164 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.165 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Start", 2020-09-12T14:18:34.165 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:48.442055+0000", 2020-09-12T14:18:34.165 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:48.442065+0000" 2020-09-12T14:18:34.165 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.165 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.165 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetInfo", 2020-09-12T14:18:34.165 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:48.442073+0000", 2020-09-12T14:18:34.166 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:48.584986+0000" 2020-09-12T14:18:34.166 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.166 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.166 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetLog", 2020-09-12T14:18:34.166 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:48.584986+0000", 2020-09-12T14:18:34.166 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:48.585067+0000" 2020-09-12T14:18:34.166 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.166 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.167 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetMissing", 2020-09-12T14:18:34.167 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:48.585067+0000", 2020-09-12T14:18:34.167 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:48.585080+0000" 2020-09-12T14:18:34.167 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.167 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.167 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/WaitUpThru", 2020-09-12T14:18:34.167 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:48.585080+0000", 2020-09-12T14:18:34.167 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:48.772619+0000" 2020-09-12T14:18:34.168 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.168 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.168 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering", 2020-09-12T14:18:34.168 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:48.442070+0000", 2020-09-12T14:18:34.168 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:48.772626+0000" 2020-09-12T14:18:34.168 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.168 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.169 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Activating", 2020-09-12T14:18:34.169 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:48.772743+0000", 2020-09-12T14:18:34.169 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:49.100274+0000" 2020-09-12T14:18:34.169 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.169 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.169 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Recovered", 2020-09-12T14:18:34.169 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:49.100274+0000", 2020-09-12T14:18:34.169 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:49.100318+0000" 2020-09-12T14:18:34.170 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.170 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.170 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Clean", 2020-09-12T14:18:34.170 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:49.100319+0000", 2020-09-12T14:18:34.170 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.100855+0000" 2020-09-12T14:18:34.170 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.170 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.170 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active", 2020-09-12T14:18:34.171 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:48.772627+0000", 2020-09-12T14:18:34.171 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.100874+0000" 2020-09-12T14:18:34.171 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.171 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.171 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary", 2020-09-12T14:18:34.171 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:48.442065+0000", 2020-09-12T14:18:34.171 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.100888+0000" 2020-09-12T14:18:34.171 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.172 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.172 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started", 2020-09-12T14:18:34.172 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:48.442043+0000", 2020-09-12T14:18:34.172 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.100891+0000" 2020-09-12T14:18:34.172 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.172 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.172 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.173 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.173 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "92", 2020-09-12T14:18:34.173 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.173 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.173 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Reset", 2020-09-12T14:18:34.173 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.100891+0000", 2020-09-12T14:18:34.173 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.101110+0000" 2020-09-12T14:18:34.173 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.174 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.174 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.174 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.174 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "98", 2020-09-12T14:18:34.174 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.174 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.175 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Start", 2020-09-12T14:18:34.175 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.101114+0000", 2020-09-12T14:18:34.175 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.101125+0000" 2020-09-12T14:18:34.175 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.175 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.175 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetInfo", 2020-09-12T14:18:34.175 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.101131+0000", 2020-09-12T14:18:34.175 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.247308+0000" 2020-09-12T14:18:34.176 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.176 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.176 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetLog", 2020-09-12T14:18:34.176 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.247309+0000", 2020-09-12T14:18:34.176 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.247407+0000" 2020-09-12T14:18:34.176 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.176 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.176 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetMissing", 2020-09-12T14:18:34.177 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.247407+0000", 2020-09-12T14:18:34.177 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.247430+0000" 2020-09-12T14:18:34.177 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.177 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.177 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/WaitUpThru", 2020-09-12T14:18:34.177 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.247431+0000", 2020-09-12T14:18:34.177 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.612051+0000" 2020-09-12T14:18:34.178 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.178 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.178 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering", 2020-09-12T14:18:34.178 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.101128+0000", 2020-09-12T14:18:34.178 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.612058+0000" 2020-09-12T14:18:34.178 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.178 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.179 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Activating", 2020-09-12T14:18:34.179 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.612121+0000", 2020-09-12T14:18:34.179 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.696246+0000" 2020-09-12T14:18:34.179 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.179 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.179 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Recovered", 2020-09-12T14:18:34.179 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.696247+0000", 2020-09-12T14:18:34.179 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.696330+0000" 2020-09-12T14:18:34.180 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.180 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.180 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Clean", 2020-09-12T14:18:34.180 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.696331+0000", 2020-09-12T14:18:34.180 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:15:55.824466+0000" 2020-09-12T14:18:34.180 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.180 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.181 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active", 2020-09-12T14:18:34.181 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.612058+0000", 2020-09-12T14:18:34.181 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:15:55.824491+0000" 2020-09-12T14:18:34.181 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.181 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.181 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary", 2020-09-12T14:18:34.181 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.101125+0000", 2020-09-12T14:18:34.181 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:15:55.824555+0000" 2020-09-12T14:18:34.182 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.182 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.182 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started", 2020-09-12T14:18:34.182 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.101111+0000", 2020-09-12T14:18:34.182 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:15:55.824559+0000" 2020-09-12T14:18:34.182 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.182 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.182 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.183 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.183 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "99", 2020-09-12T14:18:34.183 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.183 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.183 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Reset", 2020-09-12T14:18:34.183 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:15:55.824560+0000", 2020-09-12T14:18:34.183 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:15:55.824960+0000" 2020-09-12T14:18:34.184 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.184 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.184 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.184 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.184 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "100", 2020-09-12T14:18:34.184 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.184 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.184 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Start", 2020-09-12T14:18:34.185 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:15:55.824966+0000", 2020-09-12T14:18:34.185 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:15:55.824978+0000" 2020-09-12T14:18:34.185 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.185 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.185 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetInfo", 2020-09-12T14:18:34.185 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:15:55.824988+0000", 2020-09-12T14:18:34.185 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:15:55.825496+0000" 2020-09-12T14:18:34.185 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.186 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.186 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetLog", 2020-09-12T14:18:34.186 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:15:55.825496+0000", 2020-09-12T14:18:34.186 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:15:55.825558+0000" 2020-09-12T14:18:34.186 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.186 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.186 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering", 2020-09-12T14:18:34.187 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:15:55.824984+0000", 2020-09-12T14:18:34.187 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:15:55.825565+0000" 2020-09-12T14:18:34.187 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.187 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.187 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/WaitActingChange", 2020-09-12T14:18:34.187 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:15:55.825566+0000", 2020-09-12T14:18:34.187 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:15:56.101031+0000" 2020-09-12T14:18:34.187 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.188 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.188 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary", 2020-09-12T14:18:34.188 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:15:55.824979+0000", 2020-09-12T14:18:34.188 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:15:56.101050+0000" 2020-09-12T14:18:34.188 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.188 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.188 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started", 2020-09-12T14:18:34.188 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:15:55.824961+0000", 2020-09-12T14:18:34.189 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:15:56.101054+0000" 2020-09-12T14:18:34.189 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.189 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.189 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.189 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.189 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "100", 2020-09-12T14:18:34.189 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.189 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.190 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Reset", 2020-09-12T14:18:34.190 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:15:56.101054+0000", 2020-09-12T14:18:34.190 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:15:56.101280+0000" 2020-09-12T14:18:34.190 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.190 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.190 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.190 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.191 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "115", 2020-09-12T14:18:34.191 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.191 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.191 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Start", 2020-09-12T14:18:34.191 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:15:56.101284+0000", 2020-09-12T14:18:34.191 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:15:56.101296+0000" 2020-09-12T14:18:34.191 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.191 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.192 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetInfo", 2020-09-12T14:18:34.192 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:15:56.101304+0000", 2020-09-12T14:18:34.192 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:15:56.218439+0000" 2020-09-12T14:18:34.192 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.192 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.192 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetLog", 2020-09-12T14:18:34.192 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:15:56.218440+0000", 2020-09-12T14:18:34.192 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:15:56.218522+0000" 2020-09-12T14:18:34.193 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.193 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.193 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetMissing", 2020-09-12T14:18:34.193 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:15:56.218522+0000", 2020-09-12T14:18:34.193 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:15:56.218541+0000" 2020-09-12T14:18:34.193 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.193 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.194 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/WaitUpThru", 2020-09-12T14:18:34.194 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:15:56.218541+0000", 2020-09-12T14:18:34.194 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:15:56.427456+0000" 2020-09-12T14:18:34.194 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.194 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.194 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering", 2020-09-12T14:18:34.194 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:15:56.101300+0000", 2020-09-12T14:18:34.194 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:15:56.427468+0000" 2020-09-12T14:18:34.195 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.195 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.195 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Activating", 2020-09-12T14:18:34.195 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:15:56.427735+0000", 2020-09-12T14:18:34.195 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:15:57.364111+0000" 2020-09-12T14:18:34.195 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.195 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.195 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/WaitLocalBackfillReserved", 2020-09-12T14:18:34.196 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:15:57.364112+0000", 2020-09-12T14:18:34.196 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:15:57.364271+0000" 2020-09-12T14:18:34.196 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.196 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.196 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/WaitRemoteBackfillReserved", 2020-09-12T14:18:34.196 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:15:57.364272+0000", 2020-09-12T14:18:34.196 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:15:57.364692+0000" 2020-09-12T14:18:34.197 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.197 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.197 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Backfilling", 2020-09-12T14:18:34.197 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:15:57.364693+0000", 2020-09-12T14:18:34.197 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:29.532257+0000" 2020-09-12T14:18:34.197 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.197 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.197 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/NotBackfilling", 2020-09-12T14:18:34.198 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:29.532258+0000", 2020-09-12T14:18:34.198 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:29.532418+0000" 2020-09-12T14:18:34.198 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.198 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.198 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/WaitLocalBackfillReserved", 2020-09-12T14:18:34.198 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:29.532418+0000", 2020-09-12T14:18:34.198 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:42.252701+0000" 2020-09-12T14:18:34.198 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.199 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.199 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/WaitRemoteBackfillReserved", 2020-09-12T14:18:34.199 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:42.252702+0000", 2020-09-12T14:18:34.199 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:42.253081+0000" 2020-09-12T14:18:34.199 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.199 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.199 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Backfilling", 2020-09-12T14:18:34.199 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:42.253081+0000", 2020-09-12T14:18:34.200 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:17:20.034305+0000" 2020-09-12T14:18:34.200 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.200 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.200 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Recovered", 2020-09-12T14:18:34.200 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:17:20.034306+0000", 2020-09-12T14:18:34.200 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:17:20.034491+0000" 2020-09-12T14:18:34.200 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.201 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.201 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Clean", 2020-09-12T14:18:34.201 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:17:20.034491+0000", 2020-09-12T14:18:34.201 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:17:20.414809+0000" 2020-09-12T14:18:34.201 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.201 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.201 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active", 2020-09-12T14:18:34.201 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:15:56.427468+0000", 2020-09-12T14:18:34.202 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:17:20.414836+0000" 2020-09-12T14:18:34.202 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.202 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.202 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary", 2020-09-12T14:18:34.202 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:15:56.101296+0000", 2020-09-12T14:18:34.202 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:17:20.414855+0000" 2020-09-12T14:18:34.202 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.202 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.203 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started", 2020-09-12T14:18:34.203 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:15:56.101280+0000", 2020-09-12T14:18:34.203 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:17:20.414859+0000" 2020-09-12T14:18:34.203 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.203 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.203 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.203 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.203 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "115", 2020-09-12T14:18:34.204 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.204 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.204 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Reset", 2020-09-12T14:18:34.204 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:17:20.414859+0000", 2020-09-12T14:18:34.204 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:17:20.415124+0000" 2020-09-12T14:18:34.204 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.204 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.204 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.205 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.205 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.205 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.205 INFO:tasks.workunit.client.0.smithi027.stdout: "pg": "3.0", 2020-09-12T14:18:34.205 INFO:tasks.workunit.client.0.smithi027.stdout: "currently": "Started/Primary/Active/Clean", 2020-09-12T14:18:34.205 INFO:tasks.workunit.client.0.smithi027.stdout: "history": [ 2020-09-12T14:18:34.205 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.206 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "42", 2020-09-12T14:18:34.206 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.206 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.206 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Start", 2020-09-12T14:18:34.206 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:36.024269+0000", 2020-09-12T14:18:34.206 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:36.024278+0000" 2020-09-12T14:18:34.206 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.206 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.207 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetInfo", 2020-09-12T14:18:34.207 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:36.024285+0000", 2020-09-12T14:18:34.207 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:36.250426+0000" 2020-09-12T14:18:34.207 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.207 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.207 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetLog", 2020-09-12T14:18:34.207 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:36.250428+0000", 2020-09-12T14:18:34.207 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:36.250494+0000" 2020-09-12T14:18:34.208 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.208 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.208 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetMissing", 2020-09-12T14:18:34.208 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:36.250494+0000", 2020-09-12T14:18:34.208 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:36.250505+0000" 2020-09-12T14:18:34.208 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.208 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.208 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/WaitUpThru", 2020-09-12T14:18:34.209 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:36.250506+0000", 2020-09-12T14:18:34.209 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:36.342699+0000" 2020-09-12T14:18:34.209 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.209 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.209 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering", 2020-09-12T14:18:34.209 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:36.024281+0000", 2020-09-12T14:18:34.210 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:36.342706+0000" 2020-09-12T14:18:34.210 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.210 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.210 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Activating", 2020-09-12T14:18:34.210 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:36.342878+0000", 2020-09-12T14:18:34.210 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:36.820663+0000" 2020-09-12T14:18:34.210 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.211 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.211 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Recovered", 2020-09-12T14:18:34.211 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:36.820664+0000", 2020-09-12T14:18:34.211 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:36.820761+0000" 2020-09-12T14:18:34.211 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.211 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.211 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Clean", 2020-09-12T14:18:34.211 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:36.820761+0000", 2020-09-12T14:18:34.212 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:37.439198+0000" 2020-09-12T14:18:34.212 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.212 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.212 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active", 2020-09-12T14:18:34.212 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:36.342707+0000", 2020-09-12T14:18:34.212 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:37.439231+0000" 2020-09-12T14:18:34.212 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.212 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.213 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary", 2020-09-12T14:18:34.213 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:36.024279+0000", 2020-09-12T14:18:34.213 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:37.439260+0000" 2020-09-12T14:18:34.213 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.213 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.213 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started", 2020-09-12T14:18:34.213 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:36.024266+0000", 2020-09-12T14:18:34.214 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:37.439267+0000" 2020-09-12T14:18:34.214 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.214 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.214 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.214 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.214 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "42", 2020-09-12T14:18:34.214 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.215 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.215 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Reset", 2020-09-12T14:18:34.215 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:37.439267+0000", 2020-09-12T14:18:34.215 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:37.439626+0000" 2020-09-12T14:18:34.215 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.215 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.215 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.215 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.216 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "89", 2020-09-12T14:18:34.216 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.216 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.216 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Start", 2020-09-12T14:18:34.216 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:37.439635+0000", 2020-09-12T14:18:34.216 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:37.439657+0000" 2020-09-12T14:18:34.216 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.216 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.217 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetInfo", 2020-09-12T14:18:34.217 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:37.439671+0000", 2020-09-12T14:18:34.217 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:37.657721+0000" 2020-09-12T14:18:34.217 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.217 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.217 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetLog", 2020-09-12T14:18:34.217 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:37.657722+0000", 2020-09-12T14:18:34.218 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:37.657832+0000" 2020-09-12T14:18:34.218 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.218 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.218 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetMissing", 2020-09-12T14:18:34.218 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:37.657832+0000", 2020-09-12T14:18:34.218 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:37.657853+0000" 2020-09-12T14:18:34.218 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.218 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.219 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/WaitUpThru", 2020-09-12T14:18:34.219 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:37.657853+0000", 2020-09-12T14:18:34.219 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:37.998379+0000" 2020-09-12T14:18:34.219 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.219 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.219 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering", 2020-09-12T14:18:34.219 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:37.439665+0000", 2020-09-12T14:18:34.220 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:37.998393+0000" 2020-09-12T14:18:34.220 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.220 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.220 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Activating", 2020-09-12T14:18:34.220 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:37.998639+0000", 2020-09-12T14:18:34.220 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:38.336498+0000" 2020-09-12T14:18:34.220 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.220 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.221 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Recovered", 2020-09-12T14:18:34.221 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:38.336498+0000", 2020-09-12T14:18:34.221 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:38.336600+0000" 2020-09-12T14:18:34.221 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.221 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.221 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Clean", 2020-09-12T14:18:34.221 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:38.336600+0000", 2020-09-12T14:18:34.221 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.101986+0000" 2020-09-12T14:18:34.222 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.222 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.222 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active", 2020-09-12T14:18:34.222 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:37.998394+0000", 2020-09-12T14:18:34.222 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.102002+0000" 2020-09-12T14:18:34.222 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.222 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.223 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary", 2020-09-12T14:18:34.223 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:37.439657+0000", 2020-09-12T14:18:34.223 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.102012+0000" 2020-09-12T14:18:34.223 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.223 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.223 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started", 2020-09-12T14:18:34.223 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:37.439627+0000", 2020-09-12T14:18:34.223 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.102015+0000" 2020-09-12T14:18:34.224 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.224 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.224 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.224 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.224 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "92", 2020-09-12T14:18:34.224 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.224 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.224 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Reset", 2020-09-12T14:18:34.225 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.102016+0000", 2020-09-12T14:18:34.225 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.102234+0000" 2020-09-12T14:18:34.225 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.225 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.225 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.225 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.225 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "106", 2020-09-12T14:18:34.225 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.226 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.226 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Start", 2020-09-12T14:18:34.226 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.102239+0000", 2020-09-12T14:18:34.226 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.102248+0000" 2020-09-12T14:18:34.226 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.226 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.226 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetInfo", 2020-09-12T14:18:34.226 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.102256+0000", 2020-09-12T14:18:34.227 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.285615+0000" 2020-09-12T14:18:34.227 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.227 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.227 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetLog", 2020-09-12T14:18:34.227 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.285615+0000", 2020-09-12T14:18:34.227 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.285724+0000" 2020-09-12T14:18:34.227 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.227 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.228 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetMissing", 2020-09-12T14:18:34.228 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.285724+0000", 2020-09-12T14:18:34.228 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.285743+0000" 2020-09-12T14:18:34.228 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.228 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.228 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/WaitUpThru", 2020-09-12T14:18:34.228 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.285744+0000", 2020-09-12T14:18:34.229 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.612288+0000" 2020-09-12T14:18:34.229 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.229 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.229 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering", 2020-09-12T14:18:34.229 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.102253+0000", 2020-09-12T14:18:34.229 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.612296+0000" 2020-09-12T14:18:34.229 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.229 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.230 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Activating", 2020-09-12T14:18:34.230 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.612349+0000", 2020-09-12T14:18:34.230 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.841087+0000" 2020-09-12T14:18:34.230 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.230 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.230 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Recovered", 2020-09-12T14:18:34.230 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.841087+0000", 2020-09-12T14:18:34.230 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:14:17.841124+0000" 2020-09-12T14:18:34.231 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.231 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.231 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Clean", 2020-09-12T14:18:34.231 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.841124+0000", 2020-09-12T14:18:34.231 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:15.698395+0000" 2020-09-12T14:18:34.231 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.231 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.231 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active", 2020-09-12T14:18:34.232 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.612296+0000", 2020-09-12T14:18:34.232 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:15.698427+0000" 2020-09-12T14:18:34.232 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.232 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.232 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary", 2020-09-12T14:18:34.232 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.102248+0000", 2020-09-12T14:18:34.232 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:15.698489+0000" 2020-09-12T14:18:34.232 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.233 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.233 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started", 2020-09-12T14:18:34.233 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:14:17.102234+0000", 2020-09-12T14:18:34.233 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:15.698493+0000" 2020-09-12T14:18:34.233 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.233 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.233 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.234 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.234 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "106", 2020-09-12T14:18:34.234 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.234 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.234 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Reset", 2020-09-12T14:18:34.234 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:15.698494+0000", 2020-09-12T14:18:34.234 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:15.698840+0000" 2020-09-12T14:18:34.234 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.235 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.235 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.235 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.235 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "109", 2020-09-12T14:18:34.235 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.235 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.235 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Start", 2020-09-12T14:18:34.235 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:15.698847+0000", 2020-09-12T14:18:34.236 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:15.698861+0000" 2020-09-12T14:18:34.236 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.236 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.236 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetInfo", 2020-09-12T14:18:34.236 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:15.698869+0000", 2020-09-12T14:18:34.236 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:15.815895+0000" 2020-09-12T14:18:34.236 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.236 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.237 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetLog", 2020-09-12T14:18:34.237 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:15.815896+0000", 2020-09-12T14:18:34.237 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:15.815950+0000" 2020-09-12T14:18:34.237 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.237 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.237 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering", 2020-09-12T14:18:34.237 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:15.698865+0000", 2020-09-12T14:18:34.237 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:15.815958+0000" 2020-09-12T14:18:34.238 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.238 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.238 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/WaitActingChange", 2020-09-12T14:18:34.238 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:15.815958+0000", 2020-09-12T14:18:34.238 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:16.352115+0000" 2020-09-12T14:18:34.238 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.238 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.239 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary", 2020-09-12T14:18:34.239 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:15.698861+0000", 2020-09-12T14:18:34.239 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:16.352136+0000" 2020-09-12T14:18:34.239 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.239 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.239 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started", 2020-09-12T14:18:34.239 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:15.698841+0000", 2020-09-12T14:18:34.239 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:16.352140+0000" 2020-09-12T14:18:34.240 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.240 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.240 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.240 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.240 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "109", 2020-09-12T14:18:34.240 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.240 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.240 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Reset", 2020-09-12T14:18:34.241 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:16.352141+0000", 2020-09-12T14:18:34.241 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:16.352369+0000" 2020-09-12T14:18:34.241 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.241 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.241 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.241 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.241 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "117", 2020-09-12T14:18:34.241 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.242 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.242 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Start", 2020-09-12T14:18:34.242 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:16.352375+0000", 2020-09-12T14:18:34.242 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:16.352389+0000" 2020-09-12T14:18:34.242 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.242 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.242 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetInfo", 2020-09-12T14:18:34.242 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:16.352400+0000", 2020-09-12T14:18:34.243 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:16.352980+0000" 2020-09-12T14:18:34.243 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.243 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.243 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetLog", 2020-09-12T14:18:34.243 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:16.352980+0000", 2020-09-12T14:18:34.243 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:16.353057+0000" 2020-09-12T14:18:34.243 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.244 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.244 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetMissing", 2020-09-12T14:18:34.244 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:16.353058+0000", 2020-09-12T14:18:34.244 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:16.353076+0000" 2020-09-12T14:18:34.244 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.244 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.244 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/WaitUpThru", 2020-09-12T14:18:34.244 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:16.353076+0000", 2020-09-12T14:18:34.245 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:16.595148+0000" 2020-09-12T14:18:34.245 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.245 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.245 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering", 2020-09-12T14:18:34.245 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:16.352396+0000", 2020-09-12T14:18:34.245 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:16.595160+0000" 2020-09-12T14:18:34.245 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.245 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.246 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Activating", 2020-09-12T14:18:34.246 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:16.595442+0000", 2020-09-12T14:18:34.246 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:17.106190+0000" 2020-09-12T14:18:34.246 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.246 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.246 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/WaitLocalBackfillReserved", 2020-09-12T14:18:34.246 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:17.106190+0000", 2020-09-12T14:18:34.246 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:29.532332+0000" 2020-09-12T14:18:34.247 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.247 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.247 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/WaitRemoteBackfillReserved", 2020-09-12T14:18:34.247 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:29.532333+0000", 2020-09-12T14:18:34.247 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:29.532744+0000" 2020-09-12T14:18:34.247 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.247 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.248 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Backfilling", 2020-09-12T14:18:34.248 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:29.532745+0000", 2020-09-12T14:18:34.248 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:42.252630+0000" 2020-09-12T14:18:34.248 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.248 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.248 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/NotBackfilling", 2020-09-12T14:18:34.248 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:42.252631+0000", 2020-09-12T14:18:34.248 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:42.252781+0000" 2020-09-12T14:18:34.249 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.249 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.249 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/WaitLocalBackfillReserved", 2020-09-12T14:18:34.249 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:42.252781+0000", 2020-09-12T14:18:34.249 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:17:20.034700+0000" 2020-09-12T14:18:34.249 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.249 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.249 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/WaitRemoteBackfillReserved", 2020-09-12T14:18:34.250 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:17:20.034700+0000", 2020-09-12T14:18:34.250 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:17:20.035183+0000" 2020-09-12T14:18:34.250 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.250 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.250 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Backfilling", 2020-09-12T14:18:34.250 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:17:20.035184+0000", 2020-09-12T14:18:34.250 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:17:52.366634+0000" 2020-09-12T14:18:34.250 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.251 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.251 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Recovered", 2020-09-12T14:18:34.251 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:17:52.366635+0000", 2020-09-12T14:18:34.251 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:17:52.366804+0000" 2020-09-12T14:18:34.251 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.251 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.251 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Clean", 2020-09-12T14:18:34.251 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:17:52.366804+0000", 2020-09-12T14:18:34.252 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:17:53.085686+0000" 2020-09-12T14:18:34.252 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.252 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.252 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active", 2020-09-12T14:18:34.252 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:16.595161+0000", 2020-09-12T14:18:34.252 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:17:53.085714+0000" 2020-09-12T14:18:34.252 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.252 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.253 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary", 2020-09-12T14:18:34.253 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:16.352390+0000", 2020-09-12T14:18:34.253 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:17:53.085732+0000" 2020-09-12T14:18:34.253 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.253 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.253 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started", 2020-09-12T14:18:34.253 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:16.352369+0000", 2020-09-12T14:18:34.253 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:17:53.085737+0000" 2020-09-12T14:18:34.254 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.254 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.254 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.254 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.254 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "117", 2020-09-12T14:18:34.254 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.254 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.255 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Reset", 2020-09-12T14:18:34.255 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:17:53.085738+0000", 2020-09-12T14:18:34.255 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:17:53.086036+0000" 2020-09-12T14:18:34.255 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.255 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.255 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.255 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.255 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.256 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.256 INFO:tasks.workunit.client.0.smithi027.stdout: "pg": "1.0", 2020-09-12T14:18:34.256 INFO:tasks.workunit.client.0.smithi027.stdout: "currently": "Started/Primary/Active/Clean", 2020-09-12T14:18:34.256 INFO:tasks.workunit.client.0.smithi027.stdout: "history": [ 2020-09-12T14:18:34.256 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.256 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "27", 2020-09-12T14:18:34.256 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.256 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.257 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Initial", 2020-09-12T14:18:34.257 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:31.325161+0000", 2020-09-12T14:18:34.257 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:31.325282+0000" 2020-09-12T14:18:34.257 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.257 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.257 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.257 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.257 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "27", 2020-09-12T14:18:34.258 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.258 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.258 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Reset", 2020-09-12T14:18:34.258 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:31.325283+0000", 2020-09-12T14:18:34.258 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:31.325313+0000" 2020-09-12T14:18:34.258 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.258 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.258 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.259 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.259 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "29", 2020-09-12T14:18:34.259 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.259 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.259 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Start", 2020-09-12T14:18:34.259 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:31.325317+0000", 2020-09-12T14:18:34.259 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:31.325326+0000" 2020-09-12T14:18:34.259 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.260 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.260 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetInfo", 2020-09-12T14:18:34.260 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:31.325333+0000", 2020-09-12T14:18:34.260 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:31.326005+0000" 2020-09-12T14:18:34.260 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.260 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.260 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetLog", 2020-09-12T14:18:34.260 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:31.326006+0000", 2020-09-12T14:18:34.261 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:31.326082+0000" 2020-09-12T14:18:34.261 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.261 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.261 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetMissing", 2020-09-12T14:18:34.261 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:31.326082+0000", 2020-09-12T14:18:34.261 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:31.326096+0000" 2020-09-12T14:18:34.261 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.262 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.262 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/WaitUpThru", 2020-09-12T14:18:34.262 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:31.326097+0000", 2020-09-12T14:18:34.262 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:31.685521+0000" 2020-09-12T14:18:34.262 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.262 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.262 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering", 2020-09-12T14:18:34.262 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:31.325330+0000", 2020-09-12T14:18:34.263 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:31.685529+0000" 2020-09-12T14:18:34.263 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.263 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.263 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Activating", 2020-09-12T14:18:34.263 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:31.685760+0000", 2020-09-12T14:18:34.263 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:32.290330+0000" 2020-09-12T14:18:34.263 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.263 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.264 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active", 2020-09-12T14:18:34.264 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:31.685529+0000", 2020-09-12T14:18:34.264 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:32.290353+0000" 2020-09-12T14:18:34.264 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.264 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.264 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary", 2020-09-12T14:18:34.264 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:31.325327+0000", 2020-09-12T14:18:34.264 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:32.290367+0000" 2020-09-12T14:18:34.265 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.265 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.265 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started", 2020-09-12T14:18:34.265 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:31.325314+0000", 2020-09-12T14:18:34.265 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:32.290371+0000" 2020-09-12T14:18:34.265 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.265 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.266 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.266 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.266 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "29", 2020-09-12T14:18:34.266 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.266 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.266 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Reset", 2020-09-12T14:18:34.266 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:32.290371+0000", 2020-09-12T14:18:34.266 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:32.290547+0000" 2020-09-12T14:18:34.267 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.267 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.267 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.267 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.267 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "102", 2020-09-12T14:18:34.267 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.267 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.267 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Start", 2020-09-12T14:18:34.268 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:32.290550+0000", 2020-09-12T14:18:34.268 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:32.290559+0000" 2020-09-12T14:18:34.268 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.268 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.268 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetInfo", 2020-09-12T14:18:34.268 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:32.290565+0000", 2020-09-12T14:18:34.268 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:32.462732+0000" 2020-09-12T14:18:34.269 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.269 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.269 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetLog", 2020-09-12T14:18:34.269 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:32.462733+0000", 2020-09-12T14:18:34.269 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:32.462804+0000" 2020-09-12T14:18:34.269 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.269 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.270 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetMissing", 2020-09-12T14:18:34.270 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:32.462804+0000", 2020-09-12T14:18:34.270 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:32.462815+0000" 2020-09-12T14:18:34.270 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.270 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.270 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/WaitUpThru", 2020-09-12T14:18:34.270 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:32.462815+0000", 2020-09-12T14:18:34.271 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:32.833367+0000" 2020-09-12T14:18:34.271 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.271 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.271 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering", 2020-09-12T14:18:34.271 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:32.290562+0000", 2020-09-12T14:18:34.271 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:32.833381+0000" 2020-09-12T14:18:34.271 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.271 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.272 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Activating", 2020-09-12T14:18:34.272 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:32.833627+0000", 2020-09-12T14:18:34.272 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:33.194432+0000" 2020-09-12T14:18:34.272 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.272 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.272 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Recovered", 2020-09-12T14:18:34.272 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:33.194433+0000", 2020-09-12T14:18:34.272 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:13:33.194529+0000" 2020-09-12T14:18:34.273 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.273 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.273 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Clean", 2020-09-12T14:18:34.273 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:33.194529+0000", 2020-09-12T14:18:34.273 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:06.855652+0000" 2020-09-12T14:18:34.273 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.273 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.273 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active", 2020-09-12T14:18:34.274 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:32.833381+0000", 2020-09-12T14:18:34.274 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:06.855697+0000" 2020-09-12T14:18:34.274 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.274 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.274 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary", 2020-09-12T14:18:34.274 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:32.290560+0000", 2020-09-12T14:18:34.274 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:06.855761+0000" 2020-09-12T14:18:34.275 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.275 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.275 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started", 2020-09-12T14:18:34.275 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:13:32.290547+0000", 2020-09-12T14:18:34.275 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:06.855770+0000" 2020-09-12T14:18:34.275 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.275 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.275 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.276 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.276 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "102", 2020-09-12T14:18:34.276 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.276 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.276 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Reset", 2020-09-12T14:18:34.276 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:06.855771+0000", 2020-09-12T14:18:34.276 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:06.856298+0000" 2020-09-12T14:18:34.276 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.277 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.277 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.277 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.277 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "104", 2020-09-12T14:18:34.277 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.277 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.277 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Start", 2020-09-12T14:18:34.277 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:06.856310+0000", 2020-09-12T14:18:34.278 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:06.856336+0000" 2020-09-12T14:18:34.278 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.278 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.278 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetInfo", 2020-09-12T14:18:34.278 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:06.856354+0000", 2020-09-12T14:18:34.278 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:06.973308+0000" 2020-09-12T14:18:34.278 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.279 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.279 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetLog", 2020-09-12T14:18:34.279 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:06.973308+0000", 2020-09-12T14:18:34.279 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:06.973369+0000" 2020-09-12T14:18:34.279 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.279 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.279 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering", 2020-09-12T14:18:34.279 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:06.856345+0000", 2020-09-12T14:18:34.280 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:06.973376+0000" 2020-09-12T14:18:34.280 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.280 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.280 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/WaitActingChange", 2020-09-12T14:18:34.280 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:06.973378+0000", 2020-09-12T14:18:34.280 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:07.476167+0000" 2020-09-12T14:18:34.280 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.280 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.281 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary", 2020-09-12T14:18:34.281 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:06.856337+0000", 2020-09-12T14:18:34.281 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:07.476187+0000" 2020-09-12T14:18:34.281 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.281 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.281 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started", 2020-09-12T14:18:34.281 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:06.856299+0000", 2020-09-12T14:18:34.281 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:07.476192+0000" 2020-09-12T14:18:34.282 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.282 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.282 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.282 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.282 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "104", 2020-09-12T14:18:34.282 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.282 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.283 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Reset", 2020-09-12T14:18:34.283 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:07.476193+0000", 2020-09-12T14:18:34.283 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:07.476420+0000" 2020-09-12T14:18:34.283 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.283 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.283 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.283 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.283 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "119", 2020-09-12T14:18:34.284 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.284 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.284 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Start", 2020-09-12T14:18:34.284 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:07.476426+0000", 2020-09-12T14:18:34.284 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:07.476438+0000" 2020-09-12T14:18:34.284 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.284 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.284 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetInfo", 2020-09-12T14:18:34.285 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:07.476451+0000", 2020-09-12T14:18:34.285 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:07.615000+0000" 2020-09-12T14:18:34.285 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.285 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.285 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetLog", 2020-09-12T14:18:34.285 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:07.615001+0000", 2020-09-12T14:18:34.285 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:07.615095+0000" 2020-09-12T14:18:34.285 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.286 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.286 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/GetMissing", 2020-09-12T14:18:34.286 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:07.615095+0000", 2020-09-12T14:18:34.286 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:07.615118+0000" 2020-09-12T14:18:34.286 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.286 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.286 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering/WaitUpThru", 2020-09-12T14:18:34.286 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:07.615118+0000", 2020-09-12T14:18:34.287 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:07.870262+0000" 2020-09-12T14:18:34.287 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.287 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.287 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Peering", 2020-09-12T14:18:34.287 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:07.476446+0000", 2020-09-12T14:18:34.287 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:07.870274+0000" 2020-09-12T14:18:34.287 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.288 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.288 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Activating", 2020-09-12T14:18:34.288 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:07.870644+0000", 2020-09-12T14:18:34.288 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:16:08.122244+0000" 2020-09-12T14:18:34.288 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.288 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.288 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/WaitLocalBackfillReserved", 2020-09-12T14:18:34.288 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:08.122244+0000", 2020-09-12T14:18:34.289 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:17:52.366974+0000" 2020-09-12T14:18:34.289 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.289 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.289 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/WaitRemoteBackfillReserved", 2020-09-12T14:18:34.289 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:17:52.366975+0000", 2020-09-12T14:18:34.289 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:17:52.367395+0000" 2020-09-12T14:18:34.289 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.289 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.290 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Backfilling", 2020-09-12T14:18:34.290 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:17:52.367396+0000", 2020-09-12T14:18:34.290 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:18:27.370527+0000" 2020-09-12T14:18:34.290 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.290 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.290 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Recovered", 2020-09-12T14:18:34.290 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:18:27.370527+0000", 2020-09-12T14:18:34.291 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:18:27.370723+0000" 2020-09-12T14:18:34.291 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.291 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.291 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active/Clean", 2020-09-12T14:18:34.291 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:18:27.370723+0000", 2020-09-12T14:18:34.291 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:18:27.715872+0000" 2020-09-12T14:18:34.291 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.291 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.292 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary/Active", 2020-09-12T14:18:34.292 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:07.870275+0000", 2020-09-12T14:18:34.292 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:18:27.715910+0000" 2020-09-12T14:18:34.292 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.292 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.292 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started/Primary", 2020-09-12T14:18:34.292 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:07.476439+0000", 2020-09-12T14:18:34.293 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:18:27.715945+0000" 2020-09-12T14:18:34.293 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.293 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.293 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Started", 2020-09-12T14:18:34.293 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:16:07.476421+0000", 2020-09-12T14:18:34.293 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:18:27.715955+0000" 2020-09-12T14:18:34.293 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.293 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.294 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:18:34.294 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.294 INFO:tasks.workunit.client.0.smithi027.stdout: "epoch": "119", 2020-09-12T14:18:34.294 INFO:tasks.workunit.client.0.smithi027.stdout: "states": [ 2020-09-12T14:18:34.294 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:18:34.294 INFO:tasks.workunit.client.0.smithi027.stdout: "state": "Reset", 2020-09-12T14:18:34.294 INFO:tasks.workunit.client.0.smithi027.stdout: "enter": "2020-09-12T14:18:27.715956+0000", 2020-09-12T14:18:34.295 INFO:tasks.workunit.client.0.smithi027.stdout: "exit": "2020-09-12T14:18:27.716501+0000" 2020-09-12T14:18:34.295 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.295 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.295 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.295 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.295 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:18:34.295 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:18:34.295 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:18:34.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:302: TEST_backfill_priority: '[' 0 '!=' 0 ']' 2020-09-12T14:18:34.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:306: TEST_backfill_priority: echo TEST PASSED 2020-09-12T14:18:34.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:309: TEST_backfill_priority: delete_pool test1 2020-09-12T14:18:34.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: delete_pool: local poolname=test1 2020-09-12T14:18:34.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: ceph osd pool delete test1 test1 --yes-i-really-really-mean-it 2020-09-12T14:18:34.297 INFO:tasks.workunit.client.0.smithi027.stdout:TEST PASSED 2020-09-12T14:18:34.704 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test1' does not exist 2020-09-12T14:18:34.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:310: TEST_backfill_priority: delete_pool test3 2020-09-12T14:18:34.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: delete_pool: local poolname=test3 2020-09-12T14:18:34.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: ceph osd pool delete test3 test3 --yes-i-really-really-mean-it 2020-09-12T14:18:35.248 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test3' does not exist 2020-09-12T14:18:35.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:311: TEST_backfill_priority: delete_pool test7 2020-09-12T14:18:35.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: delete_pool: local poolname=test7 2020-09-12T14:18:35.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: ceph osd pool delete test7 test7 --yes-i-really-really-mean-it 2020-09-12T14:18:35.859 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test7' does not exist 2020-09-12T14:18:35.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:312: TEST_backfill_priority: kill_daemons td/osd-backfill-prio 2020-09-12T14:18:35.869 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:18:35.869 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:18:35.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:18:35.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:18:35.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:18:35.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:18:35.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:313: TEST_backfill_priority: return 0 2020-09-12T14:18:35.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh:40: run: teardown td/osd-backfill-prio 2020-09-12T14:18:35.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-backfill-prio 2020-09-12T14:18:35.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:18:35.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-backfill-prio KILL 2020-09-12T14:18:35.979 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:18:35.980 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:18:35.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:18:35.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:18:35.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:18:35.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:18:35.985 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:18:35.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:18:35.986 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:18:35.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:18:35.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:18:35.987 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:18:35.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:18:35.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:18:35.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:18:35.990 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:18:35.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:18:35.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:18:35.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-backfill-prio 2020-09-12T14:18:36.017 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:18:36.017 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:18:36.017 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:18:36.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.73258 2020-09-12T14:18:36.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:18:36.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:18:36.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2151: main: code=0 2020-09-12T14:18:36.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2155: main: teardown td/osd-backfill-prio 0 2020-09-12T14:18:36.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-backfill-prio 2020-09-12T14:18:36.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2020-09-12T14:18:36.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-backfill-prio KILL 2020-09-12T14:18:36.019 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:18:36.020 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:18:36.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:18:36.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:18:36.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:18:36.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:18:36.021 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:18:36.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:18:36.021 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:18:36.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:18:36.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:18:36.023 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:18:36.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:18:36.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:18:36.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:18:36.025 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:18:36.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:18:36.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2020-09-12T14:18:36.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-backfill-prio 2020-09-12T14:18:36.027 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:18:36.027 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:18:36.028 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.73258 2020-09-12T14:18:36.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.73258 2020-09-12T14:18:36.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:18:36.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:18:36.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2156: main: return 0 2020-09-12T14:18:36.029 INFO:teuthology.orchestra.run:Running command with timeout 3600 2020-09-12T14:18:36.030 INFO:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2020-09-12T14:18:36.068 INFO:tasks.workunit:Running workunit osd/osd-backfill-recovery-log.sh... 2020-09-12T14:18:36.069 INFO:teuthology.orchestra.run.smithi027:workunit test osd/osd-backfill-recovery-log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f346cc73d15a568d3bec1c99a58953de76f57b85 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh 2020-09-12T14:18:36.129 INFO:tasks.workunit.client.0.smithi027.stderr:stty: 'standard input': Inappropriate ioctl for device 2020-09-12T14:18:36.135 INFO:tasks.workunit.client.0.smithi027.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2020-09-12T14:18:36.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2020-09-12T14:18:36.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2020-09-12T14:18:36.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2145: main: export PYTHONWARNINGS=ignore 2020-09-12T14:18:36.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2145: main: PYTHONWARNINGS=ignore 2020-09-12T14:18:36.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2146: main: export CEPH_CONF=/dev/null 2020-09-12T14:18:36.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2146: main: CEPH_CONF=/dev/null 2020-09-12T14:18:36.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2147: main: unset CEPH_ARGS 2020-09-12T14:18:36.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2149: main: local code 2020-09-12T14:18:36.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2150: main: run td/osd-backfill-recovery-log 2020-09-12T14:18:36.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:21: run: local dir=td/osd-backfill-recovery-log 2020-09-12T14:18:36.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:22: run: shift 2020-09-12T14:18:36.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:25: run: export CEPH_MON=127.0.0.1:7129 2020-09-12T14:18:36.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:25: run: CEPH_MON=127.0.0.1:7129 2020-09-12T14:18:36.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:26: run: export CEPH_ARGS 2020-09-12T14:18:36.138 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:27: run: uuidgen 2020-09-12T14:18:36.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:27: run: CEPH_ARGS+='--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none ' 2020-09-12T14:18:36.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:18:36.139 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:30: run: set 2020-09-12T14:18:36.139 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:30: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2020-09-12T14:18:36.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:30: run: local 'funcs=TEST_backfill_log_1 2020-09-12T14:18:36.140 INFO:tasks.workunit.client.0.smithi027.stderr:TEST_backfill_log_2 2020-09-12T14:18:36.140 INFO:tasks.workunit.client.0.smithi027.stderr:TEST_recovery_1 2020-09-12T14:18:36.140 INFO:tasks.workunit.client.0.smithi027.stderr:TEST_recovery_2' 2020-09-12T14:18:36.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:31: run: for func in $funcs 2020-09-12T14:18:36.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:32: run: setup td/osd-backfill-recovery-log 2020-09-12T14:18:36.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-backfill-recovery-log 2020-09-12T14:18:36.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-backfill-recovery-log 2020-09-12T14:18:36.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-backfill-recovery-log 2020-09-12T14:18:36.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:18:36.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-backfill-recovery-log KILL 2020-09-12T14:18:36.141 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:18:36.141 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:18:36.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:18:36.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:18:36.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:18:36.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:18:36.142 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:18:36.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:18:36.143 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:18:36.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:18:36.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:18:36.143 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:18:36.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:18:36.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:18:36.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:18:36.144 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:18:36.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:18:36.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:18:36.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-backfill-recovery-log 2020-09-12T14:18:36.146 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:18:36.146 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:18:36.146 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:18:36.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.98804 2020-09-12T14:18:36.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:18:36.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:18:36.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-backfill-recovery-log 2020-09-12T14:18:36.148 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2020-09-12T14:18:36.148 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:18:36.148 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:18:36.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.98804 2020-09-12T14:18:36.149 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2020-09-12T14:18:36.151 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2020-09-12T14:18:36.151 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2020-09-12T14:18:36.151 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2020-09-12T14:18:36.151 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-backfill-recovery-log 1' TERM HUP INT 2020-09-12T14:18:36.151 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:33: run: TEST_backfill_log_1 td/osd-backfill-recovery-log 2020-09-12T14:18:36.151 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:105: TEST_backfill_log_1: local dir=td/osd-backfill-recovery-log 2020-09-12T14:18:36.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:107: TEST_backfill_log_1: _common_test td/osd-backfill-recovery-log '--osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10' 2 8 150 2020-09-12T14:18:36.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:40: _common_test: local dir=td/osd-backfill-recovery-log 2020-09-12T14:18:36.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:41: _common_test: local 'extra_opts=--osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10' 2020-09-12T14:18:36.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:42: _common_test: local loglen=2 2020-09-12T14:18:36.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:43: _common_test: local dupslen=8 2020-09-12T14:18:36.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:44: _common_test: local objects=150 2020-09-12T14:18:36.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:45: _common_test: local moreobjects=0 2020-09-12T14:18:36.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:47: _common_test: local OSDS=6 2020-09-12T14:18:36.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:49: _common_test: run_mon td/osd-backfill-recovery-log a 2020-09-12T14:18:36.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-backfill-recovery-log 2020-09-12T14:18:36.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2020-09-12T14:18:36.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2020-09-12T14:18:36.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2020-09-12T14:18:36.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-backfill-recovery-log/a 2020-09-12T14:18:36.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-backfill-recovery-log/a --run-dir=td/osd-backfill-recovery-log 2020-09-12T14:18:36.448 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2020-09-12T14:18:36.449 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:18:36.449 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:18:36.449 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:18:36.449 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:18:36.449 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:18:36.449 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:18:36.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-backfill-recovery-log/a '--log-file=td/osd-backfill-recovery-log/$name.log' '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --mon-cluster-log-file=td/osd-backfill-recovery-log/log --run-dir=td/osd-backfill-recovery-log '--pid-file=td/osd-backfill-recovery-log/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 2020-09-12T14:18:36.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: cat 2020-09-12T14:18:36.643 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a fsid 2020-09-12T14:18:36.643 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:18:36.643 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:18:36.644 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=fsid 2020-09-12T14:18:36.644 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .fsid 2020-09-12T14:18:36.644 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:18:36.644 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:18:36.644 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:18:36.644 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:18:36.644 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:18:36.645 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:18:36.645 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.98804/ceph-mon.a.asok 2020-09-12T14:18:36.645 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:18:36.645 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.98804/ceph-mon.a.asok config get fsid 2020-09-12T14:18:36.751 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a mon_host 2020-09-12T14:18:36.752 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:18:36.752 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:18:36.752 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=mon_host 2020-09-12T14:18:36.752 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .mon_host 2020-09-12T14:18:36.753 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:18:36.753 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:18:36.753 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:18:36.753 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:18:36.754 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:18:36.754 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:18:36.754 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.98804/ceph-mon.a.asok 2020-09-12T14:18:36.755 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:18:36.755 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.98804/ceph-mon.a.asok config get mon_host 2020-09-12T14:18:36.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:50: _common_test: run_mgr td/osd-backfill-recovery-log x 2020-09-12T14:18:36.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: local dir=td/osd-backfill-recovery-log 2020-09-12T14:18:36.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: shift 2020-09-12T14:18:36.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: local id=x 2020-09-12T14:18:36.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: shift 2020-09-12T14:18:36.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: local data=td/osd-backfill-recovery-log/x 2020-09-12T14:18:36.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:558: run_mgr: ceph config set mgr mgr/devicehealth/enable_monitoring off --force 2020-09-12T14:18:37.764 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: get_asok_path 2020-09-12T14:18:37.764 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:18:37.764 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:18:37.764 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:18:37.765 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:18:37.765 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:18:37.765 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:18:37.765 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:18:37.766 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-backfill-recovery-log/x '--log-file=td/osd-backfill-recovery-log/$name.log' '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --run-dir=td/osd-backfill-recovery-log '--pid-file=td/osd-backfill-recovery-log/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:18:37.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:51: _common_test: export CEPH_ARGS 2020-09-12T14:18:37.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:52: _common_test: export 'EXTRA_OPTS= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10' 2020-09-12T14:18:37.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:52: _common_test: EXTRA_OPTS=' --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10' 2020-09-12T14:18:37.788 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: expr 6 - 1 2020-09-12T14:18:37.789 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: seq 0 5 2020-09-12T14:18:37.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:18:37.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 0 2020-09-12T14:18:37.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:18:37.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:18:37.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=0 2020-09-12T14:18:37.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:18:37.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/0 2020-09-12T14:18:37.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:18:37.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:18:37.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:18:37.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:18:37.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/0' 2020-09-12T14:18:37.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/0/journal' 2020-09-12T14:18:37.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:18:37.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10' 2020-09-12T14:18:37.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:18:37.794 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:18:37.794 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:18:37.794 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:18:37.794 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:18:37.794 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:18:37.794 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:18:37.794 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:18:37.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:18:37.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:18:37.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:18:37.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:18:37.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:18:37.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:18:37.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:18:37.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:18:37.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:18:37.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:18:37.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:18:37.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/0 2020-09-12T14:18:37.797 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:18:37.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=939f8e50-94ef-4597-9942-0a84ce786b53 2020-09-12T14:18:37.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd0 939f8e50-94ef-4597-9942-0a84ce786b53' 2020-09-12T14:18:37.797 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:18:37.797 INFO:tasks.workunit.client.0.smithi027.stdout:add osd0 939f8e50-94ef-4597-9942-0a84ce786b53 2020-09-12T14:18:37.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQC92FxflXkYMBAAcMAYF0eoGWCt158084nnZg== 2020-09-12T14:18:37.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQC92FxflXkYMBAAcMAYF0eoGWCt158084nnZg=="}' 2020-09-12T14:18:37.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 939f8e50-94ef-4597-9942-0a84ce786b53 -i td/osd-backfill-recovery-log/0/new.json 2020-09-12T14:18:38.178 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:18:38.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/0/new.json 2020-09-12T14:18:38.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 0 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/0 --osd-journal=td/osd-backfill-recovery-log/0/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQC92FxflXkYMBAAcMAYF0eoGWCt158084nnZg== --osd-uuid 939f8e50-94ef-4597-9942-0a84ce786b53 2020-09-12T14:18:38.215 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:38.214+0000 7fe57725aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:18:38.216 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:38.216+0000 7fe57725aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:18:38.218 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:38.218+0000 7fe57725aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:18:38.219 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:38.218+0000 7fe57725aec0 -1 bluestore(td/osd-backfill-recovery-log/0/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/0/block: (2) No such file or directory 2020-09-12T14:18:38.219 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:38.218+0000 7fe57725aec0 -1 bluestore(td/osd-backfill-recovery-log/0/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/0/block: (2) No such file or directory 2020-09-12T14:18:38.219 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:38.218+0000 7fe57725aec0 -1 bluestore(td/osd-backfill-recovery-log/0/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/0/block: (2) No such file or directory 2020-09-12T14:18:38.258 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:38.258+0000 7fe57725aec0 -1 bluestore(td/osd-backfill-recovery-log/0) _read_fsid unparsable uuid 2020-09-12T14:18:42.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/0/keyring 2020-09-12T14:18:42.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:18:42.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd0 key to auth repository 2020-09-12T14:18:42.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:18:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd0 key to auth repository 2020-09-12T14:18:43.285 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.0 2020-09-12T14:18:43.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.0 2020-09-12T14:18:43.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 0 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/0 --osd-journal=td/osd-backfill-recovery-log/0/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:18:43.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:18:43.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:18:43.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:18:43.309 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:43.308+0000 7f4ae3ed7ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:18:43.347 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:43.347+0000 7f4ae3ed7ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:18:43.348 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:43.348+0000 7f4ae3ed7ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:18:43.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 0 2020-09-12T14:18:43.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:18:43.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T14:18:43.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:18:43.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:18:43.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:18:43.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:18:43.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:18:43.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:18:43.690 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:18:43.869 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:43.868+0000 7f4ae3ed7ec0 -1 Falling back to public interface 2020-09-12T14:18:44.094 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:18:44.712 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:44.712+0000 7f4ae3ed7ec0 -1 osd.0 0 log_to_monitors {default=true} 2020-09-12T14:18:45.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:18:45.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:18:45.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:18:45.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:18:45.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:18:45.097 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:18:45.523 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/99665,v1:127.0.0.1:6803/99665] [v2:127.0.0.1:6804/99665,v1:127.0.0.1:6805/99665] exists,up 939f8e50-94ef-4597-9942-0a84ce786b53 2020-09-12T14:18:45.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:18:45.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:18:45.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:18:45.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:18:45.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 1 2020-09-12T14:18:45.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:18:45.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:18:45.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=1 2020-09-12T14:18:45.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:18:45.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/1 2020-09-12T14:18:45.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:18:45.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:18:45.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:18:45.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:18:45.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/1' 2020-09-12T14:18:45.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/1/journal' 2020-09-12T14:18:45.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:18:45.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10' 2020-09-12T14:18:45.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:18:45.526 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:18:45.526 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:18:45.527 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:18:45.527 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:18:45.527 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:18:45.527 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:18:45.527 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:18:45.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:18:45.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:18:45.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:18:45.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:18:45.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:18:45.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:18:45.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:18:45.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:18:45.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:18:45.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:18:45.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:18:45.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/1 2020-09-12T14:18:45.529 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:18:45.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=94f04144-7980-4457-8bb0-f6155caf6267 2020-09-12T14:18:45.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd1 94f04144-7980-4457-8bb0-f6155caf6267' 2020-09-12T14:18:45.530 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:18:45.530 INFO:tasks.workunit.client.0.smithi027.stdout:add osd1 94f04144-7980-4457-8bb0-f6155caf6267 2020-09-12T14:18:45.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDF2Fxf3rogIBAANwp9cYy8uJJ518F0uzHK2Q== 2020-09-12T14:18:45.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDF2Fxf3rogIBAANwp9cYy8uJJ518F0uzHK2Q=="}' 2020-09-12T14:18:45.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 94f04144-7980-4457-8bb0-f6155caf6267 -i td/osd-backfill-recovery-log/1/new.json 2020-09-12T14:18:46.171 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:18:46.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/1/new.json 2020-09-12T14:18:46.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 1 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/1 --osd-journal=td/osd-backfill-recovery-log/1/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDF2Fxf3rogIBAANwp9cYy8uJJ518F0uzHK2Q== --osd-uuid 94f04144-7980-4457-8bb0-f6155caf6267 2020-09-12T14:18:46.209 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:46.209+0000 7f1f8633eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:18:46.211 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:46.210+0000 7f1f8633eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:18:46.211 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:46.211+0000 7f1f8633eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:18:46.212 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:46.212+0000 7f1f8633eec0 -1 bluestore(td/osd-backfill-recovery-log/1/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/1/block: (2) No such file or directory 2020-09-12T14:18:46.212 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:46.212+0000 7f1f8633eec0 -1 bluestore(td/osd-backfill-recovery-log/1/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/1/block: (2) No such file or directory 2020-09-12T14:18:46.213 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:46.212+0000 7f1f8633eec0 -1 bluestore(td/osd-backfill-recovery-log/1/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/1/block: (2) No such file or directory 2020-09-12T14:18:46.271 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:46.271+0000 7f1f8633eec0 -1 bluestore(td/osd-backfill-recovery-log/1) _read_fsid unparsable uuid 2020-09-12T14:18:50.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/1/keyring 2020-09-12T14:18:50.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:18:50.940 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd1 key to auth repository 2020-09-12T14:18:50.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd1 key to auth repository 2020-09-12T14:18:50.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:18:51.490 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T14:18:51.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.1 2020-09-12T14:18:51.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 1 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/1 --osd-journal=td/osd-backfill-recovery-log/1/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:18:51.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:18:51.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:18:51.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:18:51.514 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:51.514+0000 7f4f18552ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:18:51.548 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:51.547+0000 7f4f18552ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:18:51.549 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:51.548+0000 7f4f18552ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:18:51.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 1 2020-09-12T14:18:51.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:18:51.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:18:51.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:18:51.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:18:51.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:18:51.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:18:51.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:18:51.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:18:51.960 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:18:52.067 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:52.067+0000 7f4f18552ec0 -1 Falling back to public interface 2020-09-12T14:18:52.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:18:52.907 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:52.907+0000 7f4f18552ec0 -1 osd.1 0 log_to_monitors {default=true} 2020-09-12T14:18:53.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:18:53.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:18:53.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:18:53.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:18:53.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:18:53.370 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:18:53.810 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/100659,v1:127.0.0.1:6811/100659] [v2:127.0.0.1:6812/100659,v1:127.0.0.1:6813/100659] exists,up 94f04144-7980-4457-8bb0-f6155caf6267 2020-09-12T14:18:53.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:18:53.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:18:53.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:18:53.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:18:53.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 2 2020-09-12T14:18:53.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:18:53.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:18:53.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=2 2020-09-12T14:18:53.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:18:53.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/2 2020-09-12T14:18:53.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:18:53.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:18:53.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:18:53.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:18:53.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/2' 2020-09-12T14:18:53.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/2/journal' 2020-09-12T14:18:53.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:18:53.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10' 2020-09-12T14:18:53.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:18:53.813 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:18:53.813 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:18:53.814 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:18:53.814 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:18:53.814 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:18:53.814 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:18:53.815 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:18:53.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:18:53.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:18:53.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:18:53.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:18:53.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:18:53.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:18:53.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:18:53.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:18:53.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:18:53.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:18:53.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:18:53.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/2 2020-09-12T14:18:53.817 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:18:53.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=4fb91175-6b4c-4599-af10-b1a5b4c91904 2020-09-12T14:18:53.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd2 4fb91175-6b4c-4599-af10-b1a5b4c91904' 2020-09-12T14:18:53.818 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:18:53.818 INFO:tasks.workunit.client.0.smithi027.stdout:add osd2 4fb91175-6b4c-4599-af10-b1a5b4c91904 2020-09-12T14:18:53.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDN2FxfV6RcMRAAiaw4/1rwpUwz020HsHWIVQ== 2020-09-12T14:18:53.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDN2FxfV6RcMRAAiaw4/1rwpUwz020HsHWIVQ=="}' 2020-09-12T14:18:53.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 4fb91175-6b4c-4599-af10-b1a5b4c91904 -i td/osd-backfill-recovery-log/2/new.json 2020-09-12T14:18:54.430 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:18:54.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/2/new.json 2020-09-12T14:18:54.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 2 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/2 --osd-journal=td/osd-backfill-recovery-log/2/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDN2FxfV6RcMRAAiaw4/1rwpUwz020HsHWIVQ== --osd-uuid 4fb91175-6b4c-4599-af10-b1a5b4c91904 2020-09-12T14:18:54.469 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:54.468+0000 7ff63739bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:18:54.470 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:54.470+0000 7ff63739bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:18:54.472 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:54.471+0000 7ff63739bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:18:54.473 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:54.472+0000 7ff63739bec0 -1 bluestore(td/osd-backfill-recovery-log/2/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/2/block: (2) No such file or directory 2020-09-12T14:18:54.473 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:54.472+0000 7ff63739bec0 -1 bluestore(td/osd-backfill-recovery-log/2/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/2/block: (2) No such file or directory 2020-09-12T14:18:54.473 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:54.472+0000 7ff63739bec0 -1 bluestore(td/osd-backfill-recovery-log/2/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/2/block: (2) No such file or directory 2020-09-12T14:18:54.506 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:54.506+0000 7ff63739bec0 -1 bluestore(td/osd-backfill-recovery-log/2) _read_fsid unparsable uuid 2020-09-12T14:18:59.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/2/keyring 2020-09-12T14:18:59.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:18:59.306 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd2 key to auth repository 2020-09-12T14:18:59.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd2 key to auth repository 2020-09-12T14:18:59.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:18:59.875 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.2 2020-09-12T14:18:59.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.2 2020-09-12T14:18:59.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 2 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/2 --osd-journal=td/osd-backfill-recovery-log/2/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:18:59.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:18:59.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:18:59.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:18:59.899 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:59.897+0000 7f1253debec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:18:59.900 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:59.899+0000 7f1253debec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:18:59.901 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:18:59.900+0000 7f1253debec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:19:00.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 2 2020-09-12T14:19:00.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:19:00.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T14:19:00.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:19:00.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:19:00.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:19:00.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:19:00.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:19:00.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:19:00.369 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:19:00.421 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:00.420+0000 7f1253debec0 -1 Falling back to public interface 2020-09-12T14:19:00.766 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:19:01.340 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:01.339+0000 7f1253debec0 -1 osd.2 0 log_to_monitors {default=true} 2020-09-12T14:19:01.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:19:01.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:19:01.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:19:01.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:19:01.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:19:01.769 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:19:02.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:19:03.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:19:03.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:19:03.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 2 2020-09-12T14:19:03.333 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:19:03.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:19:03.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:19:03.791 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/101661,v1:127.0.0.1:6819/101661] [v2:127.0.0.1:6820/101661,v1:127.0.0.1:6821/101661] exists,up 4fb91175-6b4c-4599-af10-b1a5b4c91904 2020-09-12T14:19:03.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:19:03.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:19:03.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:19:03.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:19:03.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 3 2020-09-12T14:19:03.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:19:03.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:19:03.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=3 2020-09-12T14:19:03.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:19:03.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/3 2020-09-12T14:19:03.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:19:03.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:19:03.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:19:03.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:19:03.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/3' 2020-09-12T14:19:03.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/3/journal' 2020-09-12T14:19:03.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:19:03.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10' 2020-09-12T14:19:03.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:19:03.794 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:19:03.794 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:19:03.794 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:19:03.796 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:19:03.796 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:19:03.796 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:19:03.796 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:19:03.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:19:03.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:19:03.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:19:03.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:19:03.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:19:03.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:19:03.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:19:03.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:19:03.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:19:03.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:19:03.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:19:03.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/3 2020-09-12T14:19:03.798 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:19:03.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=6dbe32d4-61ca-46c8-aa5d-759edbf5cf32 2020-09-12T14:19:03.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd3 6dbe32d4-61ca-46c8-aa5d-759edbf5cf32' 2020-09-12T14:19:03.799 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:19:03.799 INFO:tasks.workunit.client.0.smithi027.stdout:add osd3 6dbe32d4-61ca-46c8-aa5d-759edbf5cf32 2020-09-12T14:19:03.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDX2FxfwY8/MBAAmomYvfbivMowkCCvSNIDDQ== 2020-09-12T14:19:03.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDX2FxfwY8/MBAAmomYvfbivMowkCCvSNIDDQ=="}' 2020-09-12T14:19:03.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 6dbe32d4-61ca-46c8-aa5d-759edbf5cf32 -i td/osd-backfill-recovery-log/3/new.json 2020-09-12T14:19:04.459 INFO:tasks.workunit.client.0.smithi027.stdout:3 2020-09-12T14:19:04.470 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/3/new.json 2020-09-12T14:19:04.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 3 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/3 --osd-journal=td/osd-backfill-recovery-log/3/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDX2FxfwY8/MBAAmomYvfbivMowkCCvSNIDDQ== --osd-uuid 6dbe32d4-61ca-46c8-aa5d-759edbf5cf32 2020-09-12T14:19:04.496 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:04.495+0000 7fef18910ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:19:04.498 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:04.496+0000 7fef18910ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:19:04.499 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:04.498+0000 7fef18910ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:19:04.500 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:04.499+0000 7fef18910ec0 -1 bluestore(td/osd-backfill-recovery-log/3/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/3/block: (2) No such file or directory 2020-09-12T14:19:04.500 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:04.499+0000 7fef18910ec0 -1 bluestore(td/osd-backfill-recovery-log/3/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/3/block: (2) No such file or directory 2020-09-12T14:19:04.500 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:04.499+0000 7fef18910ec0 -1 bluestore(td/osd-backfill-recovery-log/3/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/3/block: (2) No such file or directory 2020-09-12T14:19:04.624 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:04.623+0000 7fef18910ec0 -1 bluestore(td/osd-backfill-recovery-log/3) _read_fsid unparsable uuid 2020-09-12T14:19:09.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/3/keyring 2020-09-12T14:19:09.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:19:09.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd3 key to auth repository 2020-09-12T14:19:09.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:19:09.194 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd3 key to auth repository 2020-09-12T14:19:09.769 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.3 2020-09-12T14:19:09.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.3 2020-09-12T14:19:09.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:19:09.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 3 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/3 --osd-journal=td/osd-backfill-recovery-log/3/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:19:09.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:19:09.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:19:09.793 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:09.791+0000 7f59f7973ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:19:09.797 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:09.793+0000 7f59f7973ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:19:09.798 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:09.794+0000 7f59f7973ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:19:10.172 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 3 2020-09-12T14:19:10.172 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:19:10.172 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=3 2020-09-12T14:19:10.172 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:19:10.172 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:19:10.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:19:10.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:19:10.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:19:10.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.3 up' 2020-09-12T14:19:10.173 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:19:10.319 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:10.317+0000 7f59f7973ec0 -1 Falling back to public interface 2020-09-12T14:19:10.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:19:11.157 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:11.156+0000 7f59f7973ec0 -1 osd.3 0 log_to_monitors {default=true} 2020-09-12T14:19:11.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:19:11.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:19:11.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:19:11.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:19:11.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.3 up' 2020-09-12T14:19:11.593 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:19:12.095 INFO:tasks.workunit.client.0.smithi027.stdout:osd.3 up in weight 1 up_from 20 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/102681,v1:127.0.0.1:6827/102681] [v2:127.0.0.1:6828/102681,v1:127.0.0.1:6829/102681] exists,up 6dbe32d4-61ca-46c8-aa5d-759edbf5cf32 2020-09-12T14:19:12.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:19:12.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:19:12.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:19:12.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:19:12.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 4 2020-09-12T14:19:12.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:19:12.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:19:12.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=4 2020-09-12T14:19:12.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:19:12.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/4 2020-09-12T14:19:12.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:19:12.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:19:12.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:19:12.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:19:12.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/4' 2020-09-12T14:19:12.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/4/journal' 2020-09-12T14:19:12.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:19:12.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10' 2020-09-12T14:19:12.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:19:12.098 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:19:12.098 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:19:12.098 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:19:12.100 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:19:12.100 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:19:12.101 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:19:12.101 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:19:12.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:19:12.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:19:12.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:19:12.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:19:12.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:19:12.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:19:12.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:19:12.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:19:12.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:19:12.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:19:12.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:19:12.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/4 2020-09-12T14:19:12.103 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:19:12.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=c3ada37c-4bd9-4887-bcd0-e37e4ad3762a 2020-09-12T14:19:12.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd4 c3ada37c-4bd9-4887-bcd0-e37e4ad3762a' 2020-09-12T14:19:12.103 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:19:12.103 INFO:tasks.workunit.client.0.smithi027.stdout:add osd4 c3ada37c-4bd9-4887-bcd0-e37e4ad3762a 2020-09-12T14:19:12.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDg2Fxfc43BBhAA9+2ZIuTk8oJTRu2wQpMu9g== 2020-09-12T14:19:12.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDg2Fxfc43BBhAA9+2ZIuTk8oJTRu2wQpMu9g=="}' 2020-09-12T14:19:12.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new c3ada37c-4bd9-4887-bcd0-e37e4ad3762a -i td/osd-backfill-recovery-log/4/new.json 2020-09-12T14:19:12.683 INFO:tasks.workunit.client.0.smithi027.stdout:4 2020-09-12T14:19:12.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/4/new.json 2020-09-12T14:19:12.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 4 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/4 --osd-journal=td/osd-backfill-recovery-log/4/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDg2Fxfc43BBhAA9+2ZIuTk8oJTRu2wQpMu9g== --osd-uuid c3ada37c-4bd9-4887-bcd0-e37e4ad3762a 2020-09-12T14:19:12.720 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:12.719+0000 7f5fff8f8ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:19:12.722 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:12.721+0000 7f5fff8f8ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:19:12.723 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:12.722+0000 7f5fff8f8ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:19:12.723 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:12.722+0000 7f5fff8f8ec0 -1 bluestore(td/osd-backfill-recovery-log/4/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/4/block: (2) No such file or directory 2020-09-12T14:19:12.723 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:12.722+0000 7f5fff8f8ec0 -1 bluestore(td/osd-backfill-recovery-log/4/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/4/block: (2) No such file or directory 2020-09-12T14:19:12.724 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:12.722+0000 7f5fff8f8ec0 -1 bluestore(td/osd-backfill-recovery-log/4/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/4/block: (2) No such file or directory 2020-09-12T14:19:12.805 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:12.804+0000 7f5fff8f8ec0 -1 bluestore(td/osd-backfill-recovery-log/4) _read_fsid unparsable uuid 2020-09-12T14:19:17.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/4/keyring 2020-09-12T14:19:17.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:19:17.401 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd4 key to auth repository 2020-09-12T14:19:17.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd4 key to auth repository 2020-09-12T14:19:17.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:19:17.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.4 2020-09-12T14:19:17.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 4 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/4 --osd-journal=td/osd-backfill-recovery-log/4/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:19:17.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:19:17.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:19:17.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:19:17.946 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.4 2020-09-12T14:19:17.968 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:17.968+0000 7face8302ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:19:18.002 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:18.001+0000 7face8302ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:19:18.003 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:18.002+0000 7face8302ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:19:18.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 4 2020-09-12T14:19:18.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:19:18.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=4 2020-09-12T14:19:18.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:19:18.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:19:18.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:19:18.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:19:18.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:19:18.353 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.4 up' 2020-09-12T14:19:18.353 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:19:18.523 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:18.522+0000 7face8302ec0 -1 Falling back to public interface 2020-09-12T14:19:18.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:19:19.370 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:19.370+0000 7face8302ec0 -1 osd.4 0 log_to_monitors {default=true} 2020-09-12T14:19:19.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:19:19.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:19:19.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:19:19.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:19:19.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.4 up' 2020-09-12T14:19:19.804 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:19:20.270 INFO:tasks.workunit.client.0.smithi027.stdout:osd.4 up in weight 1 up_from 25 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/103675,v1:127.0.0.1:6835/103675] [v2:127.0.0.1:6836/103675,v1:127.0.0.1:6837/103675] exists,up c3ada37c-4bd9-4887-bcd0-e37e4ad3762a 2020-09-12T14:19:20.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:19:20.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:19:20.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:19:20.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:19:20.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 5 2020-09-12T14:19:20.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:19:20.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:19:20.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=5 2020-09-12T14:19:20.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:19:20.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/5 2020-09-12T14:19:20.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:19:20.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:19:20.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:19:20.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:19:20.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/5' 2020-09-12T14:19:20.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/5/journal' 2020-09-12T14:19:20.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:19:20.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10' 2020-09-12T14:19:20.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:19:20.273 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:19:20.273 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:19:20.273 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:19:20.275 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:19:20.275 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:19:20.275 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:19:20.276 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:19:20.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:19:20.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:19:20.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:19:20.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:19:20.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:19:20.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:19:20.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:19:20.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:19:20.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:19:20.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:19:20.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:19:20.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/5 2020-09-12T14:19:20.277 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:19:20.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=d8980100-82c3-4f1d-a54c-df6b7ae95352 2020-09-12T14:19:20.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd5 d8980100-82c3-4f1d-a54c-df6b7ae95352' 2020-09-12T14:19:20.278 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:19:20.278 INFO:tasks.workunit.client.0.smithi027.stdout:add osd5 d8980100-82c3-4f1d-a54c-df6b7ae95352 2020-09-12T14:19:20.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDo2Fxf3CUtERAAawLkPWDzgSHTN/ZsiHaCCg== 2020-09-12T14:19:20.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDo2Fxf3CUtERAAawLkPWDzgSHTN/ZsiHaCCg=="}' 2020-09-12T14:19:20.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new d8980100-82c3-4f1d-a54c-df6b7ae95352 -i td/osd-backfill-recovery-log/5/new.json 2020-09-12T14:19:20.919 INFO:tasks.workunit.client.0.smithi027.stdout:5 2020-09-12T14:19:20.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/5/new.json 2020-09-12T14:19:20.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 5 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/5 --osd-journal=td/osd-backfill-recovery-log/5/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDo2Fxf3CUtERAAawLkPWDzgSHTN/ZsiHaCCg== --osd-uuid d8980100-82c3-4f1d-a54c-df6b7ae95352 2020-09-12T14:19:20.956 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:20.956+0000 7f7afaaa8ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:19:20.958 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:20.958+0000 7f7afaaa8ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:19:20.960 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:20.959+0000 7f7afaaa8ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:19:20.960 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:20.960+0000 7f7afaaa8ec0 -1 bluestore(td/osd-backfill-recovery-log/5/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/5/block: (2) No such file or directory 2020-09-12T14:19:20.960 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:20.960+0000 7f7afaaa8ec0 -1 bluestore(td/osd-backfill-recovery-log/5/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/5/block: (2) No such file or directory 2020-09-12T14:19:20.961 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:20.960+0000 7f7afaaa8ec0 -1 bluestore(td/osd-backfill-recovery-log/5/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/5/block: (2) No such file or directory 2020-09-12T14:19:21.094 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:21.094+0000 7f7afaaa8ec0 -1 bluestore(td/osd-backfill-recovery-log/5) _read_fsid unparsable uuid 2020-09-12T14:19:26.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/5/keyring 2020-09-12T14:19:26.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:19:26.211 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd5 key to auth repository 2020-09-12T14:19:26.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd5 key to auth repository 2020-09-12T14:19:26.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:19:26.746 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.5 2020-09-12T14:19:26.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.5 2020-09-12T14:19:26.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:19:26.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:19:26.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:19:26.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 5 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/5 --osd-journal=td/osd-backfill-recovery-log/5/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:19:26.770 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:26.769+0000 7fda28fa8ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:19:26.818 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:26.818+0000 7fda28fa8ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:19:26.819 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:26.819+0000 7fda28fa8ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:19:27.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 5 2020-09-12T14:19:27.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:19:27.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=5 2020-09-12T14:19:27.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:19:27.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:19:27.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:19:27.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:19:27.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:19:27.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.5 up' 2020-09-12T14:19:27.174 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:19:27.335 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:27.335+0000 7fda28fa8ec0 -1 Falling back to public interface 2020-09-12T14:19:27.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:19:28.214 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:19:28.213+0000 7fda28fa8ec0 -1 osd.5 0 log_to_monitors {default=true} 2020-09-12T14:19:28.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:19:28.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:19:28.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:19:28.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:19:28.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.5 up' 2020-09-12T14:19:28.580 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:19:29.024 INFO:tasks.workunit.client.0.smithi027.stdout:osd.5 up in weight 1 up_from 30 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/104669,v1:127.0.0.1:6843/104669] [v2:127.0.0.1:6844/104669,v1:127.0.0.1:6845/104669] exists,up d8980100-82c3-4f1d-a54c-df6b7ae95352 2020-09-12T14:19:29.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:19:29.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:19:29.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:19:29.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:59: _common_test: create_pool test 1 1 2020-09-12T14:19:29.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test 1 1 2020-09-12T14:19:29.590 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test' already exists 2020-09-12T14:19:29.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:19:30.602 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: seq 1 150 2020-09-12T14:19:30.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:30.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-1 /etc/passwd 2020-09-12T14:19:31.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:31.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-2 /etc/passwd 2020-09-12T14:19:31.823 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:31.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-3 /etc/passwd 2020-09-12T14:19:32.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:32.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-4 /etc/passwd 2020-09-12T14:19:32.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:32.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-5 /etc/passwd 2020-09-12T14:19:32.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:32.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-6 /etc/passwd 2020-09-12T14:19:32.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:32.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-7 /etc/passwd 2020-09-12T14:19:32.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:32.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-8 /etc/passwd 2020-09-12T14:19:33.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:33.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-9 /etc/passwd 2020-09-12T14:19:33.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:33.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-10 /etc/passwd 2020-09-12T14:19:33.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:33.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-11 /etc/passwd 2020-09-12T14:19:33.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:33.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-12 /etc/passwd 2020-09-12T14:19:33.644 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:33.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-13 /etc/passwd 2020-09-12T14:19:33.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:33.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-14 /etc/passwd 2020-09-12T14:19:33.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:33.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-15 /etc/passwd 2020-09-12T14:19:34.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:34.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-16 /etc/passwd 2020-09-12T14:19:34.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:34.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-17 /etc/passwd 2020-09-12T14:19:34.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:34.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-18 /etc/passwd 2020-09-12T14:19:34.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:34.606 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-19 /etc/passwd 2020-09-12T14:19:34.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:34.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-20 /etc/passwd 2020-09-12T14:19:34.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:34.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-21 /etc/passwd 2020-09-12T14:19:35.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:35.014 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-22 /etc/passwd 2020-09-12T14:19:35.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:35.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-23 /etc/passwd 2020-09-12T14:19:35.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:35.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-24 /etc/passwd 2020-09-12T14:19:35.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:35.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-25 /etc/passwd 2020-09-12T14:19:35.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:35.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-26 /etc/passwd 2020-09-12T14:19:35.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:35.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-27 /etc/passwd 2020-09-12T14:19:35.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:35.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-28 /etc/passwd 2020-09-12T14:19:36.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:36.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-29 /etc/passwd 2020-09-12T14:19:36.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:36.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-30 /etc/passwd 2020-09-12T14:19:36.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:36.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-31 /etc/passwd 2020-09-12T14:19:36.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:36.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-32 /etc/passwd 2020-09-12T14:19:36.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:36.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-33 /etc/passwd 2020-09-12T14:19:36.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:36.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-34 /etc/passwd 2020-09-12T14:19:36.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:36.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-35 /etc/passwd 2020-09-12T14:19:37.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:37.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-36 /etc/passwd 2020-09-12T14:19:37.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:37.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-37 /etc/passwd 2020-09-12T14:19:37.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:37.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-38 /etc/passwd 2020-09-12T14:19:37.672 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:37.672 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-39 /etc/passwd 2020-09-12T14:19:37.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:37.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-40 /etc/passwd 2020-09-12T14:19:37.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:37.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-41 /etc/passwd 2020-09-12T14:19:38.116 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:38.116 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-42 /etc/passwd 2020-09-12T14:19:38.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:38.284 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-43 /etc/passwd 2020-09-12T14:19:38.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:38.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-44 /etc/passwd 2020-09-12T14:19:38.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:38.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-45 /etc/passwd 2020-09-12T14:19:38.677 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:38.678 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-46 /etc/passwd 2020-09-12T14:19:38.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:38.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-47 /etc/passwd 2020-09-12T14:19:38.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:38.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-48 /etc/passwd 2020-09-12T14:19:39.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:39.122 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-49 /etc/passwd 2020-09-12T14:19:39.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:39.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-50 /etc/passwd 2020-09-12T14:19:39.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:39.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-51 /etc/passwd 2020-09-12T14:19:39.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:39.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-52 /etc/passwd 2020-09-12T14:19:39.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:39.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-53 /etc/passwd 2020-09-12T14:19:39.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:39.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-54 /etc/passwd 2020-09-12T14:19:40.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:40.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-55 /etc/passwd 2020-09-12T14:19:40.177 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:40.178 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-56 /etc/passwd 2020-09-12T14:19:40.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:40.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-57 /etc/passwd 2020-09-12T14:19:40.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:40.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-58 /etc/passwd 2020-09-12T14:19:40.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:40.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-59 /etc/passwd 2020-09-12T14:19:40.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:40.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-60 /etc/passwd 2020-09-12T14:19:40.915 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:40.915 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-61 /etc/passwd 2020-09-12T14:19:41.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:41.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-62 /etc/passwd 2020-09-12T14:19:41.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:41.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-63 /etc/passwd 2020-09-12T14:19:41.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:41.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-64 /etc/passwd 2020-09-12T14:19:41.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:41.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-65 /etc/passwd 2020-09-12T14:19:41.677 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:41.677 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-66 /etc/passwd 2020-09-12T14:19:41.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:41.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-67 /etc/passwd 2020-09-12T14:19:41.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:41.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-68 /etc/passwd 2020-09-12T14:19:42.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:42.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-69 /etc/passwd 2020-09-12T14:19:42.298 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:42.298 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-70 /etc/passwd 2020-09-12T14:19:42.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:42.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-71 /etc/passwd 2020-09-12T14:19:42.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:42.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-72 /etc/passwd 2020-09-12T14:19:42.841 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:42.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-73 /etc/passwd 2020-09-12T14:19:43.009 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:43.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-74 /etc/passwd 2020-09-12T14:19:43.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:43.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-75 /etc/passwd 2020-09-12T14:19:43.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:43.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-76 /etc/passwd 2020-09-12T14:19:43.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:43.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-77 /etc/passwd 2020-09-12T14:19:43.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:43.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-78 /etc/passwd 2020-09-12T14:19:43.730 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:43.730 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-79 /etc/passwd 2020-09-12T14:19:43.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:43.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-80 /etc/passwd 2020-09-12T14:19:44.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:44.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-81 /etc/passwd 2020-09-12T14:19:44.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:44.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-82 /etc/passwd 2020-09-12T14:19:44.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:44.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-83 /etc/passwd 2020-09-12T14:19:44.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:44.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-84 /etc/passwd 2020-09-12T14:19:44.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:44.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-85 /etc/passwd 2020-09-12T14:19:44.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:44.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-86 /etc/passwd 2020-09-12T14:19:44.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:44.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-87 /etc/passwd 2020-09-12T14:19:45.062 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:45.062 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-88 /etc/passwd 2020-09-12T14:19:45.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:45.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-89 /etc/passwd 2020-09-12T14:19:45.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:45.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-90 /etc/passwd 2020-09-12T14:19:45.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:45.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-91 /etc/passwd 2020-09-12T14:19:45.649 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:45.649 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-92 /etc/passwd 2020-09-12T14:19:45.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:45.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-93 /etc/passwd 2020-09-12T14:19:45.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:45.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-94 /etc/passwd 2020-09-12T14:19:46.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:46.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-95 /etc/passwd 2020-09-12T14:19:46.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:46.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-96 /etc/passwd 2020-09-12T14:19:46.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:46.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-97 /etc/passwd 2020-09-12T14:19:46.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:46.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-98 /etc/passwd 2020-09-12T14:19:46.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:46.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-99 /etc/passwd 2020-09-12T14:19:46.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:46.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-100 /etc/passwd 2020-09-12T14:19:47.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:47.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-101 /etc/passwd 2020-09-12T14:19:47.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:47.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-102 /etc/passwd 2020-09-12T14:19:47.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:47.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-103 /etc/passwd 2020-09-12T14:19:47.584 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:47.584 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-104 /etc/passwd 2020-09-12T14:19:47.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:47.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-105 /etc/passwd 2020-09-12T14:19:47.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:47.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-106 /etc/passwd 2020-09-12T14:19:48.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:48.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-107 /etc/passwd 2020-09-12T14:19:48.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:48.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-108 /etc/passwd 2020-09-12T14:19:48.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:48.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-109 /etc/passwd 2020-09-12T14:19:48.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:48.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-110 /etc/passwd 2020-09-12T14:19:48.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:48.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-111 /etc/passwd 2020-09-12T14:19:48.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:48.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-112 /etc/passwd 2020-09-12T14:19:49.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:49.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-113 /etc/passwd 2020-09-12T14:19:49.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:49.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-114 /etc/passwd 2020-09-12T14:19:49.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:49.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-115 /etc/passwd 2020-09-12T14:19:49.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:49.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-116 /etc/passwd 2020-09-12T14:19:49.644 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:49.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-117 /etc/passwd 2020-09-12T14:19:49.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:49.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-118 /etc/passwd 2020-09-12T14:19:49.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:49.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-119 /etc/passwd 2020-09-12T14:19:50.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:50.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-120 /etc/passwd 2020-09-12T14:19:50.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:50.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-121 /etc/passwd 2020-09-12T14:19:50.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:50.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-122 /etc/passwd 2020-09-12T14:19:50.548 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:50.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-123 /etc/passwd 2020-09-12T14:19:50.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:50.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-124 /etc/passwd 2020-09-12T14:19:50.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:50.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-125 /etc/passwd 2020-09-12T14:19:51.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:51.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-126 /etc/passwd 2020-09-12T14:19:51.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:51.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-127 /etc/passwd 2020-09-12T14:19:51.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:51.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-128 /etc/passwd 2020-09-12T14:19:51.437 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:51.437 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-129 /etc/passwd 2020-09-12T14:19:51.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:51.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-130 /etc/passwd 2020-09-12T14:19:51.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:51.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-131 /etc/passwd 2020-09-12T14:19:51.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:51.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-132 /etc/passwd 2020-09-12T14:19:52.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:52.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-133 /etc/passwd 2020-09-12T14:19:52.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:52.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-134 /etc/passwd 2020-09-12T14:19:52.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:52.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-135 /etc/passwd 2020-09-12T14:19:52.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:52.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-136 /etc/passwd 2020-09-12T14:19:52.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:52.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-137 /etc/passwd 2020-09-12T14:19:52.848 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:52.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-138 /etc/passwd 2020-09-12T14:19:53.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:53.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-139 /etc/passwd 2020-09-12T14:19:53.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:53.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-140 /etc/passwd 2020-09-12T14:19:53.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:53.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-141 /etc/passwd 2020-09-12T14:19:53.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:53.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-142 /etc/passwd 2020-09-12T14:19:53.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:53.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-143 /etc/passwd 2020-09-12T14:19:53.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:53.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-144 /etc/passwd 2020-09-12T14:19:54.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:54.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-145 /etc/passwd 2020-09-12T14:19:54.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:54.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-146 /etc/passwd 2020-09-12T14:19:54.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:54.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-147 /etc/passwd 2020-09-12T14:19:54.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:54.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-148 /etc/passwd 2020-09-12T14:19:54.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:54.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-149 /etc/passwd 2020-09-12T14:19:54.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:19:54.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-150 /etc/passwd 2020-09-12T14:19:54.990 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:67: _common_test: ceph pg dump pgs --format=json 2020-09-12T14:19:54.991 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:67: _common_test: jq '.pg_stats[0].up[]' 2020-09-12T14:19:55.383 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:19:55.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:67: _common_test: ceph osd out 1 0 2 2020-09-12T14:19:56.020 INFO:tasks.workunit.client.0.smithi027.stderr:osd.1 is already out. osd.0 is already out. osd.2 is already out. 2020-09-12T14:19:56.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:68: _common_test: '[' 0 '!=' 0 ']' 2020-09-12T14:19:56.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:74: _common_test: sleep 1 2020-09-12T14:19:57.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:75: _common_test: wait_for_clean 2020-09-12T14:19:57.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:19:57.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:19:57.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:19:57.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:19:57.032 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 90 .1 2020-09-12T14:19:57.032 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:19:57.032 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:19:57.033 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:19:57.033 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:19:57.033 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:19:57.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:19:57.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:19:57.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:19:57.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:19:57.139 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:19:57.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:19:57.610 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:19:57.610 INFO:tasks.workunit.client.0.smithi027.stderr:2 2020-09-12T14:19:57.610 INFO:tasks.workunit.client.0.smithi027.stderr:3 2020-09-12T14:19:57.610 INFO:tasks.workunit.client.0.smithi027.stderr:4 2020-09-12T14:19:57.610 INFO:tasks.workunit.client.0.smithi027.stderr:5' 2020-09-12T14:19:57.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:19:57.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:19:57.611 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:19:57.767 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836497 2020-09-12T14:19:57.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836497 2020-09-12T14:19:57.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836497' 2020-09-12T14:19:57.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:19:57.769 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:19:57.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672975 2020-09-12T14:19:57.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672975 2020-09-12T14:19:57.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836497 1-42949672975' 2020-09-12T14:19:57.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:19:57.911 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:19:58.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509454 2020-09-12T14:19:58.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509454 2020-09-12T14:19:58.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836497 1-42949672975 2-64424509454' 2020-09-12T14:19:58.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:19:58.061 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2020-09-12T14:19:58.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=85899345933 2020-09-12T14:19:58.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 85899345933 2020-09-12T14:19:58.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836497 1-42949672975 2-64424509454 3-85899345933' 2020-09-12T14:19:58.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:19:58.202 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2020-09-12T14:19:58.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=107374182410 2020-09-12T14:19:58.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 107374182410 2020-09-12T14:19:58.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836497 1-42949672975 2-64424509454 3-85899345933 4-107374182410' 2020-09-12T14:19:58.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:19:58.344 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2020-09-12T14:19:58.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=128849018888 2020-09-12T14:19:58.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 128849018888 2020-09-12T14:19:58.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836497 1-42949672975 2-64424509454 3-85899345933 4-107374182410 5-128849018888' 2020-09-12T14:19:58.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:19:58.484 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836497 2020-09-12T14:19:58.484 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:19:58.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:19:58.486 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836497 2020-09-12T14:19:58.486 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:19:58.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836497 2020-09-12T14:19:58.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836497' 2020-09-12T14:19:58.487 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836497 2020-09-12T14:19:58.487 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:19:58.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836495 -lt 21474836497 2020-09-12T14:19:58.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:19:59.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:19:59.892 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:20:00.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836497 -lt 21474836497 2020-09-12T14:20:00.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:20:00.306 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672975 2020-09-12T14:20:00.306 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:20:00.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:20:00.308 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672975 2020-09-12T14:20:00.308 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:20:00.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672975 2020-09-12T14:20:00.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672975' 2020-09-12T14:20:00.309 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672975 2020-09-12T14:20:00.309 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:20:00.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672976 -lt 42949672975 2020-09-12T14:20:00.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:20:00.775 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509454 2020-09-12T14:20:00.775 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:20:00.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:20:00.777 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509454 2020-09-12T14:20:00.777 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:20:00.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509454 2020-09-12T14:20:00.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509454' 2020-09-12T14:20:00.779 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509454 2020-09-12T14:20:00.779 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:20:01.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509454 -lt 64424509454 2020-09-12T14:20:01.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:20:01.182 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 3-85899345933 2020-09-12T14:20:01.183 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:20:01.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=3 2020-09-12T14:20:01.184 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 3-85899345933 2020-09-12T14:20:01.184 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:20:01.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=85899345933 2020-09-12T14:20:01.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.3 seq 85899345933' 2020-09-12T14:20:01.186 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.3 seq 85899345933 2020-09-12T14:20:01.186 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 3 2020-09-12T14:20:01.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 85899345933 -lt 85899345933 2020-09-12T14:20:01.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:20:01.595 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 4-107374182410 2020-09-12T14:20:01.595 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:20:01.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=4 2020-09-12T14:20:01.597 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 4-107374182410 2020-09-12T14:20:01.597 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:20:01.598 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=107374182410 2020-09-12T14:20:01.598 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.4 seq 107374182410' 2020-09-12T14:20:01.598 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.4 seq 107374182410 2020-09-12T14:20:01.599 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 4 2020-09-12T14:20:02.032 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 107374182411 -lt 107374182410 2020-09-12T14:20:02.032 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:20:02.033 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 5-128849018888 2020-09-12T14:20:02.033 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:20:02.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=5 2020-09-12T14:20:02.035 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 5-128849018888 2020-09-12T14:20:02.035 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:20:02.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=128849018888 2020-09-12T14:20:02.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.5 seq 128849018888' 2020-09-12T14:20:02.036 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.5 seq 128849018888 2020-09-12T14:20:02.037 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 5 2020-09-12T14:20:02.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 128849018889 -lt 128849018888 2020-09-12T14:20:02.443 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:20:02.444 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:02.444 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:02.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 1 == 0 2020-09-12T14:20:02.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:02.973 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:02.973 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:02.973 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:02.973 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:02.973 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:02.974 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:03.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:03.473 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:03.473 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:03.474 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:04.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:04.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' -1 2020-09-12T14:20:04.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1598: wait_for_clean: loop=0 2020-09-12T14:20:04.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1599: wait_for_clean: num_active_clean=0 2020-09-12T14:20:04.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:04.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:04.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:04.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:04.122 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:04.122 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:04.122 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:04.122 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:04.122 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:04.123 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:04.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:04.517 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:04.517 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:04.518 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:05.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:05.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:05.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:05.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:05.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:05.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:05.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:05.150 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:05.150 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:05.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=10954 2020-09-12T14:20:05.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 10954 '!=' null 2020-09-12T14:20:05.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:05.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:05.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:05.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:05.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:05.820 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:05.820 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:05.820 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:05.820 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:05.820 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:05.820 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:06.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:06.213 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:06.214 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:06.214 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:06.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:06.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:06.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:06.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:06.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:06.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:06.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:06.747 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:06.747 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:07.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=8693 2020-09-12T14:20:07.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 8693 '!=' null 2020-09-12T14:20:07.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:07.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:07.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:07.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:07.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:07.383 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:07.383 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:07.384 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:07.384 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:07.384 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:07.384 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:07.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:07.809 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:07.810 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:07.810 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:08.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:08.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:08.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:08.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:08.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:08.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:08.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:08.428 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:08.428 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:08.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=19960 2020-09-12T14:20:08.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 19960 '!=' null 2020-09-12T14:20:08.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:08.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:08.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:09.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:09.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:09.064 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:09.064 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:09.064 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:09.065 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:09.065 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:09.065 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:09.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:09.456 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:09.456 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:09.456 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:10.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:10.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:10.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:10.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:10.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:10.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:10.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:10.100 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:10.100 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:10.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=17391 2020-09-12T14:20:10.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 17391 '!=' null 2020-09-12T14:20:10.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:10.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:10.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:10.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:10.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:10.738 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:10.738 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:10.738 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:10.738 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:10.738 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:10.738 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:11.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:11.131 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:11.132 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:11.132 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:11.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:11.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:11.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:11.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:11.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:11.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:11.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:11.675 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:11.675 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:12.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=28341 2020-09-12T14:20:12.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 28341 '!=' null 2020-09-12T14:20:12.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:12.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:12.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:12.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:12.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:12.326 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:12.326 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:12.326 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:12.326 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:12.326 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:12.327 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:12.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:12.724 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:12.725 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:12.725 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:13.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:13.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:13.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:13.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:13.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:13.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:13.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:13.257 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:13.258 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:13.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=21899 2020-09-12T14:20:13.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 21899 '!=' null 2020-09-12T14:20:13.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:13.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:13.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:13.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:13.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:13.906 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:13.906 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:13.906 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:13.906 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:13.906 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:13.907 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:14.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:14.297 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:14.298 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:14.298 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:14.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:14.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:14.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:14.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:14.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:14.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:14.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:14.826 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:14.827 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:15.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=21899 2020-09-12T14:20:15.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 21899 '!=' null 2020-09-12T14:20:15.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:15.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:15.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:15.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:15.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:15.503 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:15.503 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:15.504 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:15.504 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:15.504 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:15.504 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:15.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:15.897 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:15.897 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:15.897 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:16.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:16.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:16.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:16.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:16.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:16.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:16.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:16.426 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:16.426 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:17.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=21899 2020-09-12T14:20:17.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 21899 '!=' null 2020-09-12T14:20:17.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:17.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:17.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:17.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:17.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:17.126 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:17.126 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:17.127 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:17.127 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:17.127 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:17.127 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:17.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:17.521 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:17.521 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:17.521 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:18.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:18.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:18.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:18.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:18.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:18.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:18.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:18.144 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:18.144 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:18.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=33494 2020-09-12T14:20:18.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 33494 '!=' null 2020-09-12T14:20:18.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:18.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:18.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:18.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:18.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:18.784 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:18.784 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:18.784 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:18.784 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:18.785 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:18.785 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:19.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:19.183 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:19.184 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:19.184 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:19.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:19.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:19.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:19.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:19.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:19.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:19.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:19.723 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:19.723 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:20.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=22544 2020-09-12T14:20:20.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 22544 '!=' null 2020-09-12T14:20:20.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:20.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:20.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:20.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:20.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:20.388 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:20.388 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:20.388 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:20.389 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:20.389 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:20.389 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:20.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:20.780 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:20.780 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:20.781 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:21.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:21.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:21.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:21.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:21.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:21.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:21.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:21.340 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:21.340 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:21.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=33495 2020-09-12T14:20:21.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 33495 '!=' null 2020-09-12T14:20:21.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:21.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:21.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:21.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:21.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:21.997 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:21.997 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:21.997 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:21.997 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:21.998 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:21.998 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:22.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:22.387 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:22.388 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:22.388 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:22.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:22.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:22.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:22.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:22.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:22.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:22.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:22.951 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:22.951 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:23.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=33495 2020-09-12T14:20:23.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 33495 '!=' null 2020-09-12T14:20:23.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:23.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:23.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:23.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:23.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:23.597 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:23.597 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:23.597 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:23.597 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:23.598 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:23.598 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:23.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:23.992 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:23.993 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:23.993 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:24.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:24.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:24.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:24.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:24.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:24.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:24.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:24.523 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:24.523 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:25.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=22544 2020-09-12T14:20:25.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 22544 '!=' null 2020-09-12T14:20:25.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:25.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:25.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:25.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:25.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:25.276 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:25.276 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:25.276 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:25.276 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:25.277 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:25.277 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:25.677 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:25.679 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:25.679 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:25.679 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:26.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:26.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:26.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:26.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:26.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:26.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:26.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:26.211 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:26.211 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:26.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=22544 2020-09-12T14:20:26.765 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 22544 '!=' null 2020-09-12T14:20:26.765 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:26.765 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:26.765 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:26.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:26.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:26.866 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:26.866 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:26.867 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:26.867 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:26.867 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:26.867 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:27.262 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:27.263 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:27.263 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:27.263 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:27.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:27.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:27.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:27.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:27.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:27.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:27.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:27.840 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:27.840 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:28.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=34139 2020-09-12T14:20:28.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 34139 '!=' null 2020-09-12T14:20:28.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:28.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:28.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:28.468 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:28.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:28.470 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:28.470 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:28.470 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:28.470 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:28.470 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:28.470 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:28.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:28.867 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:28.868 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:28.868 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:29.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:29.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:29.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:29.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:29.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:29.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:29.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:29.412 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:29.412 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:29.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=22544 2020-09-12T14:20:29.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 22544 '!=' null 2020-09-12T14:20:29.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:29.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:29.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:30.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:30.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:30.041 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:30.041 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:30.041 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:30.041 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:30.041 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:30.042 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:30.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:30.431 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:30.431 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:30.432 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:30.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:30.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:30.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:30.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:30.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:30.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:30.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:30.960 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:30.960 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:31.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=22544 2020-09-12T14:20:31.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 22544 '!=' null 2020-09-12T14:20:31.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:31.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:31.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:31.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:31.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:31.592 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:31.592 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:31.592 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:31.592 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:31.592 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:31.592 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:32.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:32.008 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:32.008 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:32.008 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:32.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:32.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:32.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:32.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:32.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:32.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:32.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:32.542 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:32.542 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:33.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=34139 2020-09-12T14:20:33.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 34139 '!=' null 2020-09-12T14:20:33.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:33.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:33.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:33.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:33.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:33.244 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:33.244 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:33.244 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:33.245 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:33.245 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:33.245 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:33.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:33.644 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:33.644 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:33.645 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:34.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:34.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:34.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:34.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:34.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:34.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:34.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:34.176 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:34.176 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:34.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=23188 2020-09-12T14:20:34.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 23188 '!=' null 2020-09-12T14:20:34.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:34.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:34.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:34.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:34.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:34.826 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:34.826 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:34.826 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:34.827 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:34.827 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:34.827 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:35.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:35.219 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:35.219 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:35.219 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:35.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:35.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:35.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:35.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:35.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:35.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:35.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:35.770 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:35.771 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:36.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=23188 2020-09-12T14:20:36.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 23188 '!=' null 2020-09-12T14:20:36.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:36.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:36.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:36.431 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:36.431 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:36.432 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:36.432 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:36.432 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:36.432 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:36.432 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:36.433 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:36.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:36.828 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:36.829 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:36.829 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:37.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:37.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:37.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:37.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:37.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:37.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:37.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:37.358 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:37.359 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:37.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=34783 2020-09-12T14:20:37.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 34783 '!=' null 2020-09-12T14:20:37.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:37.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:37.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:37.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:37.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:37.990 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:37.990 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:37.990 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:37.991 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:37.991 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:37.991 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:38.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:38.387 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:38.387 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:38.388 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:38.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:38.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:38.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:38.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:38.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:38.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:38.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:38.922 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:38.922 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:39.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=34783 2020-09-12T14:20:39.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 34783 '!=' null 2020-09-12T14:20:39.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:39.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:39.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:39.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:39.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:39.575 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:39.575 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:39.575 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:39.575 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:39.575 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:39.576 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:39.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:39.970 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:39.970 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:39.971 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:40.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:40.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:40.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:40.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:40.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:40.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:40.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:40.506 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:40.506 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:41.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=23188 2020-09-12T14:20:41.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 23188 '!=' null 2020-09-12T14:20:41.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:41.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:41.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:41.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:41.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:41.225 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:41.225 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:41.225 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:41.225 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:41.226 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:41.226 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:41.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:41.709 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:41.709 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:41.710 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:42.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:42.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:42.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:42.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:42.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:42.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:42.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:42.249 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:42.249 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:42.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=27053 2020-09-12T14:20:42.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 27053 '!=' null 2020-09-12T14:20:42.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:42.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:42.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:42.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:42.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:42.891 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:42.891 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:42.891 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:42.892 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:42.892 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:42.892 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:43.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:43.281 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:43.282 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:43.282 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:43.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:43.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:43.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:43.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:43.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:43.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:43.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:43.815 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:43.816 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:44.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=33817 2020-09-12T14:20:44.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 33817 '!=' null 2020-09-12T14:20:44.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:44.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:44.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:44.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:44.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:44.447 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:44.447 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:44.447 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:44.448 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:44.448 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:44.448 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:44.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:44.837 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:44.837 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:44.837 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:45.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:45.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:45.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:45.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:45.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:45.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:45.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:45.370 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:45.370 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:45.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=16425 2020-09-12T14:20:45.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 16425 '!=' null 2020-09-12T14:20:45.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:45.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:45.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:45.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:45.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:45.997 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:45.997 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:45.997 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:45.997 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:45.997 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:45.997 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:46.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:20:46.388 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:46.389 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:46.389 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:46.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:20:46.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:20:46.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:20:46.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:20:46.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:20:46.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:20:46.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:20:46.962 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:20:46.962 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:20:47.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=16425 2020-09-12T14:20:47.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 16425 '!=' null 2020-09-12T14:20:47.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:20:47.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:20:47.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:20:47.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:20:47.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:20:47.589 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:20:47.589 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:20:47.589 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:20:47.589 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:20:47.590 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:20:47.590 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:20:47.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:20:47.983 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:20:47.983 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:20:47.984 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:20:48.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 1 2020-09-12T14:20:48.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T14:20:48.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T14:20:48.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:77: _common_test: flush_pg_stats 2020-09-12T14:20:48.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:20:48.515 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:20:48.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:20:48.922 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:20:48.922 INFO:tasks.workunit.client.0.smithi027.stderr:2 2020-09-12T14:20:48.922 INFO:tasks.workunit.client.0.smithi027.stderr:3 2020-09-12T14:20:48.922 INFO:tasks.workunit.client.0.smithi027.stderr:4 2020-09-12T14:20:48.923 INFO:tasks.workunit.client.0.smithi027.stderr:5' 2020-09-12T14:20:48.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:20:48.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:20:48.923 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:20:49.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836509 2020-09-12T14:20:49.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836509 2020-09-12T14:20:49.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836509' 2020-09-12T14:20:49.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:20:49.070 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:20:49.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672987 2020-09-12T14:20:49.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672987 2020-09-12T14:20:49.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836509 1-42949672987' 2020-09-12T14:20:49.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:20:49.211 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:20:49.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509466 2020-09-12T14:20:49.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509466 2020-09-12T14:20:49.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836509 1-42949672987 2-64424509466' 2020-09-12T14:20:49.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:20:49.351 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2020-09-12T14:20:49.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=85899345945 2020-09-12T14:20:49.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 85899345945 2020-09-12T14:20:49.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836509 1-42949672987 2-64424509466 3-85899345945' 2020-09-12T14:20:49.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:20:49.492 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2020-09-12T14:20:49.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=107374182422 2020-09-12T14:20:49.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 107374182422 2020-09-12T14:20:49.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836509 1-42949672987 2-64424509466 3-85899345945 4-107374182422' 2020-09-12T14:20:49.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:20:49.636 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2020-09-12T14:20:49.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=128849018901 2020-09-12T14:20:49.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 128849018901 2020-09-12T14:20:49.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836509 1-42949672987 2-64424509466 3-85899345945 4-107374182422 5-128849018901' 2020-09-12T14:20:49.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:20:49.777 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836509 2020-09-12T14:20:49.777 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:20:49.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:20:49.779 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836509 2020-09-12T14:20:49.779 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:20:49.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836509 2020-09-12T14:20:49.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836509' 2020-09-12T14:20:49.780 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836509 2020-09-12T14:20:49.780 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:20:50.197 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836509 -lt 21474836509 2020-09-12T14:20:50.197 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:20:50.198 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672987 2020-09-12T14:20:50.198 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:20:50.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:20:50.200 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672987 2020-09-12T14:20:50.200 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:20:50.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672987 2020-09-12T14:20:50.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672987' 2020-09-12T14:20:50.201 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672987 2020-09-12T14:20:50.201 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:20:50.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672988 -lt 42949672987 2020-09-12T14:20:50.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:20:50.615 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509466 2020-09-12T14:20:50.615 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:20:50.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:20:50.617 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509466 2020-09-12T14:20:50.617 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:20:50.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509466 2020-09-12T14:20:50.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509466' 2020-09-12T14:20:50.618 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509466 2020-09-12T14:20:50.618 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:20:51.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509466 -lt 64424509466 2020-09-12T14:20:51.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:20:51.035 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 3-85899345945 2020-09-12T14:20:51.035 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:20:51.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=3 2020-09-12T14:20:51.037 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 3-85899345945 2020-09-12T14:20:51.037 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:20:51.038 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=85899345945 2020-09-12T14:20:51.038 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.3 seq 85899345945' 2020-09-12T14:20:51.038 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.3 seq 85899345945 2020-09-12T14:20:51.039 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 3 2020-09-12T14:20:51.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 85899345945 -lt 85899345945 2020-09-12T14:20:51.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:20:51.459 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 4-107374182422 2020-09-12T14:20:51.459 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:20:51.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=4 2020-09-12T14:20:51.461 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 4-107374182422 2020-09-12T14:20:51.461 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:20:51.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=107374182422 2020-09-12T14:20:51.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.4 seq 107374182422' 2020-09-12T14:20:51.462 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.4 seq 107374182422 2020-09-12T14:20:51.462 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 4 2020-09-12T14:20:51.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 107374182423 -lt 107374182422 2020-09-12T14:20:51.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:20:51.875 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 5-128849018901 2020-09-12T14:20:51.876 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:20:51.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=5 2020-09-12T14:20:51.877 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 5-128849018901 2020-09-12T14:20:51.878 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:20:51.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=128849018901 2020-09-12T14:20:51.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.5 seq 128849018901' 2020-09-12T14:20:51.879 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.5 seq 128849018901 2020-09-12T14:20:51.879 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 5 2020-09-12T14:20:52.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 128849018901 -lt 128849018901 2020-09-12T14:20:52.283 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:79: _common_test: ceph pg dump pgs --format=json 2020-09-12T14:20:52.283 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:79: _common_test: jq '.pg_stats[0].up_primary' 2020-09-12T14:20:52.686 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:20:52.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:79: _common_test: newprimary=4 2020-09-12T14:20:52.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:80: _common_test: kill_daemons 2020-09-12T14:20:52.696 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:20:52.697 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:20:52.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:20:52.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:20:52.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:20:54.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:20:54.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:82: _common_test: ERRORS=0 2020-09-12T14:20:54.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:83: _common_test: _objectstore_tool_nodown td/osd-backfill-recovery-log 4 --no-mon-config --pgid 1.0 --op log 2020-09-12T14:20:54.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:83: _common_test: tee td/osd-backfill-recovery-log/result.log 2020-09-12T14:20:54.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1195: _objectstore_tool_nodown: local dir=td/osd-backfill-recovery-log 2020-09-12T14:20:54.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: _objectstore_tool_nodown: shift 2020-09-12T14:20:54.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: _objectstore_tool_nodown: local id=4 2020-09-12T14:20:54.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1198: _objectstore_tool_nodown: shift 2020-09-12T14:20:54.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: _objectstore_tool_nodown: local osd_data=td/osd-backfill-recovery-log/4 2020-09-12T14:20:54.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1202: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-backfill-recovery-log/4 --no-mon-config --pgid 1.0 --op log 2020-09-12T14:20:55.359 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:20:55.360 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_log_t": { 2020-09-12T14:20:55.360 INFO:tasks.workunit.client.0.smithi027.stdout: "head": "33'150", 2020-09-12T14:20:55.360 INFO:tasks.workunit.client.0.smithi027.stdout: "tail": "33'148", 2020-09-12T14:20:55.360 INFO:tasks.workunit.client.0.smithi027.stdout: "log": [ 2020-09-12T14:20:55.360 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:20:55.360 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:20:55.360 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:e9c179f9:::obj-149:head", 2020-09-12T14:20:55.361 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "33'149", 2020-09-12T14:20:55.361 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:20:55.361 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4633.0:1", 2020-09-12T14:20:55.361 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:20:55.361 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:19:54.730708+0000", 2020-09-12T14:20:55.361 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:20:55.361 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:20:55.362 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:20:55.362 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:20:55.362 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:20:55.362 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:20:55.362 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:20:55.362 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:20:55.362 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:20:55.362 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:20:55.363 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:20:55.363 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:20:55.363 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:20:55.363 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:20:55.364 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:20:55.364 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:20:55.364 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:20:55.364 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:20:55.364 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:cb39fd60:::obj-150:head", 2020-09-12T14:20:55.364 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "33'150", 2020-09-12T14:20:55.365 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:20:55.365 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4636.0:1", 2020-09-12T14:20:55.365 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:20:55.365 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:19:54.848401+0000", 2020-09-12T14:20:55.365 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:20:55.365 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:20:55.365 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:20:55.365 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:20:55.366 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:20:55.366 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:20:55.366 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:20:55.366 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:20:55.366 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:20:55.366 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:20:55.366 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:20:55.367 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:20:55.367 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:20:55.367 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:20:55.367 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:20:55.367 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:20:55.367 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:20:55.367 INFO:tasks.workunit.client.0.smithi027.stdout: "dups": [ 2020-09-12T14:20:55.367 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:20:55.368 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4609.0:1", 2020-09-12T14:20:55.368 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "33'141", 2020-09-12T14:20:55.368 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "141", 2020-09-12T14:20:55.368 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:20:55.368 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:20:55.368 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:20:55.368 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4612.0:1", 2020-09-12T14:20:55.369 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "33'142", 2020-09-12T14:20:55.369 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "142", 2020-09-12T14:20:55.369 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:20:55.369 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:20:55.369 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:20:55.369 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4615.0:1", 2020-09-12T14:20:55.369 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "33'143", 2020-09-12T14:20:55.369 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "143", 2020-09-12T14:20:55.370 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:20:55.370 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:20:55.370 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:20:55.370 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4618.0:1", 2020-09-12T14:20:55.370 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "33'144", 2020-09-12T14:20:55.370 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "144", 2020-09-12T14:20:55.370 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:20:55.371 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:20:55.371 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:20:55.371 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4621.0:1", 2020-09-12T14:20:55.371 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "33'145", 2020-09-12T14:20:55.371 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "145", 2020-09-12T14:20:55.371 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:20:55.371 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:20:55.371 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:20:55.372 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4624.0:1", 2020-09-12T14:20:55.372 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "33'146", 2020-09-12T14:20:55.372 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "146", 2020-09-12T14:20:55.372 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:20:55.372 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:20:55.372 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:20:55.372 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4627.0:1", 2020-09-12T14:20:55.373 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "33'147", 2020-09-12T14:20:55.373 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "147", 2020-09-12T14:20:55.373 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:20:55.373 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:20:55.373 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:20:55.373 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4630.0:1", 2020-09-12T14:20:55.373 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "33'148", 2020-09-12T14:20:55.373 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "148", 2020-09-12T14:20:55.374 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:20:55.374 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:20:55.374 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:20:55.374 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:20:55.374 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_missing_t": { 2020-09-12T14:20:55.374 INFO:tasks.workunit.client.0.smithi027.stdout: "missing": [], 2020-09-12T14:20:55.374 INFO:tasks.workunit.client.0.smithi027.stdout: "may_include_deletes": true 2020-09-12T14:20:55.374 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:20:55.375 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:20:55.376 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:84: _common_test: jq '.pg_log_t.log | length' td/osd-backfill-recovery-log/result.log 2020-09-12T14:20:55.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:84: _common_test: LOGLEN=2 2020-09-12T14:20:55.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:85: _common_test: '[' 2 '!=' 2 ']' 2020-09-12T14:20:55.376 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:89: _common_test: jq '.pg_log_t.dups | length' td/osd-backfill-recovery-log/result.log 2020-09-12T14:20:55.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:89: _common_test: DUPSLEN=8 2020-09-12T14:20:55.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:90: _common_test: '[' 8 '!=' 8 ']' 2020-09-12T14:20:55.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:94: _common_test: grep 'copy_up_to\|copy_after' td/osd-backfill-recovery-log/osd.0.log td/osd-backfill-recovery-log/osd.1.log td/osd-backfill-recovery-log/osd.2.log td/osd-backfill-recovery-log/osd.3.log td/osd-backfill-recovery-log/osd.4.log td/osd-backfill-recovery-log/osd.5.log 2020-09-12T14:20:55.377 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to max 2 2020-09-12T14:20:55.377 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to copy log version 33'150 2020-09-12T14:20:55.377 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to copy log version 33'149 2020-09-12T14:20:55.377 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after earliest_dup_version 141 2020-09-12T14:20:55.377 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'141 2020-09-12T14:20:55.377 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'142 2020-09-12T14:20:55.378 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'143 2020-09-12T14:20:55.378 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'144 2020-09-12T14:20:55.378 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'145 2020-09-12T14:20:55.378 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'146 2020-09-12T14:20:55.378 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'147 2020-09-12T14:20:55.378 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'148 2020-09-12T14:20:55.378 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to max 2 2020-09-12T14:20:55.379 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to copy log version 33'150 2020-09-12T14:20:55.379 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to copy log version 33'149 2020-09-12T14:20:55.379 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after earliest_dup_version 141 2020-09-12T14:20:55.379 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'141 2020-09-12T14:20:55.379 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'142 2020-09-12T14:20:55.379 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'143 2020-09-12T14:20:55.379 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'144 2020-09-12T14:20:55.379 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'145 2020-09-12T14:20:55.380 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'146 2020-09-12T14:20:55.380 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'147 2020-09-12T14:20:55.380 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'148 2020-09-12T14:20:55.380 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to max 2 2020-09-12T14:20:55.380 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to copy log version 33'150 2020-09-12T14:20:55.380 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to copy log version 33'149 2020-09-12T14:20:55.380 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after earliest_dup_version 141 2020-09-12T14:20:55.380 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'141 2020-09-12T14:20:55.381 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'142 2020-09-12T14:20:55.381 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'143 2020-09-12T14:20:55.391 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'144 2020-09-12T14:20:55.391 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'145 2020-09-12T14:20:55.392 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'146 2020-09-12T14:20:55.392 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'147 2020-09-12T14:20:55.392 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:19:57.178+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'148 2020-09-12T14:20:55.392 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:20:41.787+0000 7f4ac15d9700 20 copy_after v 33'150 2020-09-12T14:20:55.392 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:20:41.787+0000 7f4ac15d9700 20 copy_up_to/copy_after earliest_dup_version 141 2020-09-12T14:20:55.392 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:20:41.787+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'141 2020-09-12T14:20:55.392 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:20:41.787+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'142 2020-09-12T14:20:55.393 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:20:41.787+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'143 2020-09-12T14:20:55.393 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:20:41.787+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'144 2020-09-12T14:20:55.393 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:20:41.787+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'145 2020-09-12T14:20:55.393 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:20:41.787+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'146 2020-09-12T14:20:55.393 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:20:41.787+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'147 2020-09-12T14:20:55.393 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:20:41.787+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'148 2020-09-12T14:20:55.393 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:20:41.787+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'149 2020-09-12T14:20:55.394 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:20:41.787+0000 7f4ac15d9700 20 copy_up_to/copy_after copy dup from log version 33'150 2020-09-12T14:20:55.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:95: _common_test: rm -f td/osd-backfill-recovery-log/result.log 2020-09-12T14:20:55.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:96: _common_test: '[' 0 '!=' 0 ']' 2020-09-12T14:20:55.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:34: run: teardown td/osd-backfill-recovery-log 2020-09-12T14:20:55.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-backfill-recovery-log 2020-09-12T14:20:55.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:20:55.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-backfill-recovery-log KILL 2020-09-12T14:20:55.395 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:20:55.396 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:20:55.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:20:55.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:20:55.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:20:55.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:20:55.401 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:20:55.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:20:55.402 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:20:55.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:20:55.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:20:55.404 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:20:55.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:20:55.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:20:55.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:20:55.406 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:20:55.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:20:55.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:20:55.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-backfill-recovery-log 2020-09-12T14:20:55.423 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:20:55.424 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:20:55.424 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:20:55.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.98804 2020-09-12T14:20:55.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:20:55.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:20:55.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:31: run: for func in $funcs 2020-09-12T14:20:55.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:32: run: setup td/osd-backfill-recovery-log 2020-09-12T14:20:55.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-backfill-recovery-log 2020-09-12T14:20:55.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-backfill-recovery-log 2020-09-12T14:20:55.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-backfill-recovery-log 2020-09-12T14:20:55.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:20:55.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-backfill-recovery-log KILL 2020-09-12T14:20:55.426 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:20:55.426 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:20:55.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:20:55.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:20:55.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:20:55.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:20:55.428 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:20:55.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:20:55.429 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:20:55.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:20:55.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:20:55.430 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:20:55.431 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:20:55.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:20:55.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:20:55.432 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:20:55.433 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:20:55.434 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:20:55.434 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-backfill-recovery-log 2020-09-12T14:20:55.435 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:20:55.435 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:20:55.435 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:20:55.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.98804 2020-09-12T14:20:55.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:20:55.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:20:55.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-backfill-recovery-log 2020-09-12T14:20:55.437 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2020-09-12T14:20:55.437 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:20:55.438 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:20:55.438 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.98804 2020-09-12T14:20:55.439 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2020-09-12T14:20:55.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2020-09-12T14:20:55.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2020-09-12T14:20:55.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-backfill-recovery-log 1' TERM HUP INT 2020-09-12T14:20:55.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:33: run: TEST_backfill_log_2 td/osd-backfill-recovery-log 2020-09-12T14:20:55.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:113: TEST_backfill_log_2: local dir=td/osd-backfill-recovery-log 2020-09-12T14:20:55.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:115: TEST_backfill_log_2: _common_test td/osd-backfill-recovery-log '--osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2' 2 148 150 2020-09-12T14:20:55.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:40: _common_test: local dir=td/osd-backfill-recovery-log 2020-09-12T14:20:55.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:41: _common_test: local 'extra_opts=--osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2' 2020-09-12T14:20:55.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:42: _common_test: local loglen=2 2020-09-12T14:20:55.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:43: _common_test: local dupslen=148 2020-09-12T14:20:55.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:44: _common_test: local objects=150 2020-09-12T14:20:55.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:45: _common_test: local moreobjects=0 2020-09-12T14:20:55.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:47: _common_test: local OSDS=6 2020-09-12T14:20:55.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:49: _common_test: run_mon td/osd-backfill-recovery-log a 2020-09-12T14:20:55.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-backfill-recovery-log 2020-09-12T14:20:55.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2020-09-12T14:20:55.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2020-09-12T14:20:55.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2020-09-12T14:20:55.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-backfill-recovery-log/a 2020-09-12T14:20:55.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-backfill-recovery-log/a --run-dir=td/osd-backfill-recovery-log 2020-09-12T14:20:55.766 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2020-09-12T14:20:55.767 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:20:55.767 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:20:55.767 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:20:55.767 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:20:55.767 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:20:55.768 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:20:55.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-backfill-recovery-log/a '--log-file=td/osd-backfill-recovery-log/$name.log' '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --mon-cluster-log-file=td/osd-backfill-recovery-log/log --run-dir=td/osd-backfill-recovery-log '--pid-file=td/osd-backfill-recovery-log/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 2020-09-12T14:20:56.009 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: cat 2020-09-12T14:20:56.010 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a fsid 2020-09-12T14:20:56.011 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:20:56.011 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:20:56.011 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=fsid 2020-09-12T14:20:56.011 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .fsid 2020-09-12T14:20:56.012 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:20:56.012 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:20:56.012 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:20:56.012 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:20:56.012 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:20:56.013 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:20:56.013 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.98804/ceph-mon.a.asok 2020-09-12T14:20:56.013 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:20:56.013 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.98804/ceph-mon.a.asok config get fsid 2020-09-12T14:20:56.124 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a mon_host 2020-09-12T14:20:56.125 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:20:56.125 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:20:56.125 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=mon_host 2020-09-12T14:20:56.125 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .mon_host 2020-09-12T14:20:56.126 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:20:56.126 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:20:56.126 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:20:56.126 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:20:56.127 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:20:56.127 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:20:56.127 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.98804/ceph-mon.a.asok 2020-09-12T14:20:56.127 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:20:56.127 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.98804/ceph-mon.a.asok config get mon_host 2020-09-12T14:20:56.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:50: _common_test: run_mgr td/osd-backfill-recovery-log x 2020-09-12T14:20:56.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: local dir=td/osd-backfill-recovery-log 2020-09-12T14:20:56.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: shift 2020-09-12T14:20:56.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: local id=x 2020-09-12T14:20:56.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: shift 2020-09-12T14:20:56.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: local data=td/osd-backfill-recovery-log/x 2020-09-12T14:20:56.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:558: run_mgr: ceph config set mgr mgr/devicehealth/enable_monitoring off --force 2020-09-12T14:20:57.183 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: get_asok_path 2020-09-12T14:20:57.183 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:20:57.184 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:20:57.184 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:20:57.184 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:20:57.184 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:20:57.184 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:20:57.185 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:20:57.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: ceph-mgr --id x --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd_pg_log_dups_tracked=10 --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-backfill-recovery-log/x '--log-file=td/osd-backfill-recovery-log/$name.log' '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --run-dir=td/osd-backfill-recovery-log '--pid-file=td/osd-backfill-recovery-log/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:20:57.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:51: _common_test: export CEPH_ARGS 2020-09-12T14:20:57.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:52: _common_test: export 'EXTRA_OPTS= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2' 2020-09-12T14:20:57.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:52: _common_test: EXTRA_OPTS=' --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2' 2020-09-12T14:20:57.209 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: expr 6 - 1 2020-09-12T14:20:57.210 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: seq 0 5 2020-09-12T14:20:57.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:20:57.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 0 2020-09-12T14:20:57.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:20:57.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:20:57.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=0 2020-09-12T14:20:57.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:20:57.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/0 2020-09-12T14:20:57.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:20:57.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:20:57.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:20:57.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:20:57.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/0' 2020-09-12T14:20:57.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/0/journal' 2020-09-12T14:20:57.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:20:57.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2' 2020-09-12T14:20:57.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:20:57.215 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:20:57.215 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:20:57.215 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:20:57.216 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:20:57.216 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:20:57.216 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:20:57.216 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:20:57.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:20:57.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:20:57.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:20:57.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:20:57.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:20:57.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:20:57.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:20:57.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:20:57.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:20:57.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:20:57.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:20:57.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/0 2020-09-12T14:20:57.218 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:20:57.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=467179b5-0e95-493e-9de8-242ab7f80e00 2020-09-12T14:20:57.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd0 467179b5-0e95-493e-9de8-242ab7f80e00' 2020-09-12T14:20:57.219 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:20:57.219 INFO:tasks.workunit.client.0.smithi027.stdout:add osd0 467179b5-0e95-493e-9de8-242ab7f80e00 2020-09-12T14:20:57.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQBJ2VxfatOmDRAAPYxucR548njgxk9aqB9+mw== 2020-09-12T14:20:57.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQBJ2VxfatOmDRAAPYxucR548njgxk9aqB9+mw=="}' 2020-09-12T14:20:57.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 467179b5-0e95-493e-9de8-242ab7f80e00 -i td/osd-backfill-recovery-log/0/new.json 2020-09-12T14:20:57.570 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:20:57.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/0/new.json 2020-09-12T14:20:57.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 0 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/0 --osd-journal=td/osd-backfill-recovery-log/0/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQBJ2VxfatOmDRAAPYxucR548njgxk9aqB9+mw== --osd-uuid 467179b5-0e95-493e-9de8-242ab7f80e00 2020-09-12T14:20:57.607 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:20:57.606+0000 7f0b0f3abec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:20:57.609 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:20:57.607+0000 7f0b0f3abec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:20:57.610 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:20:57.608+0000 7f0b0f3abec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:20:57.610 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:20:57.609+0000 7f0b0f3abec0 -1 bluestore(td/osd-backfill-recovery-log/0/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/0/block: (2) No such file or directory 2020-09-12T14:20:57.610 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:20:57.609+0000 7f0b0f3abec0 -1 bluestore(td/osd-backfill-recovery-log/0/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/0/block: (2) No such file or directory 2020-09-12T14:20:57.610 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:20:57.609+0000 7f0b0f3abec0 -1 bluestore(td/osd-backfill-recovery-log/0/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/0/block: (2) No such file or directory 2020-09-12T14:20:57.725 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:20:57.724+0000 7f0b0f3abec0 -1 bluestore(td/osd-backfill-recovery-log/0) _read_fsid unparsable uuid 2020-09-12T14:21:02.342 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/0/keyring 2020-09-12T14:21:02.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:21:02.343 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd0 key to auth repository 2020-09-12T14:21:02.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd0 key to auth repository 2020-09-12T14:21:02.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:21:02.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.0 2020-09-12T14:21:02.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 0 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/0 --osd-journal=td/osd-backfill-recovery-log/0/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:21:02.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:21:02.883 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.0 2020-09-12T14:21:02.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:21:02.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:21:02.905 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:02.904+0000 7f74cd4dcec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:02.932 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:02.931+0000 7f74cd4dcec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:02.933 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:02.932+0000 7f74cd4dcec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:03.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 0 2020-09-12T14:21:03.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:21:03.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T14:21:03.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:21:03.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:21:03.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:21:03.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:21:03.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:21:03.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:21:03.371 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:21:03.450 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:03.448+0000 7f74cd4dcec0 -1 Falling back to public interface 2020-09-12T14:21:03.767 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:21:04.275 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:04.273+0000 7f74cd4dcec0 -1 osd.0 0 log_to_monitors {default=true} 2020-09-12T14:21:04.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:21:04.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:21:04.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:21:04.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:21:04.770 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:21:04.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:21:05.251 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/112470,v1:127.0.0.1:6803/112470] [v2:127.0.0.1:6804/112470,v1:127.0.0.1:6805/112470] exists,up 467179b5-0e95-493e-9de8-242ab7f80e00 2020-09-12T14:21:05.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:21:05.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:21:05.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:21:05.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:21:05.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 1 2020-09-12T14:21:05.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:21:05.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:21:05.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=1 2020-09-12T14:21:05.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:21:05.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/1 2020-09-12T14:21:05.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:21:05.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:21:05.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:21:05.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:21:05.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/1' 2020-09-12T14:21:05.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/1/journal' 2020-09-12T14:21:05.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:21:05.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2' 2020-09-12T14:21:05.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:21:05.254 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:21:05.254 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:21:05.255 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:21:05.255 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:21:05.255 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:21:05.255 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:21:05.255 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:21:05.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:21:05.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:21:05.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:21:05.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:21:05.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:21:05.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:21:05.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:21:05.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:21:05.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:21:05.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:21:05.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:21:05.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/1 2020-09-12T14:21:05.257 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:21:05.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=0c1e3a09-3808-4473-abe2-bae793ea6138 2020-09-12T14:21:05.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd1 0c1e3a09-3808-4473-abe2-bae793ea6138' 2020-09-12T14:21:05.258 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:21:05.258 INFO:tasks.workunit.client.0.smithi027.stdout:add osd1 0c1e3a09-3808-4473-abe2-bae793ea6138 2020-09-12T14:21:05.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQBR2Vxf3pLsDxAAUA8UEKp2wxNKvcUDIw9JyA== 2020-09-12T14:21:05.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQBR2Vxf3pLsDxAAUA8UEKp2wxNKvcUDIw9JyA=="}' 2020-09-12T14:21:05.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 0c1e3a09-3808-4473-abe2-bae793ea6138 -i td/osd-backfill-recovery-log/1/new.json 2020-09-12T14:21:05.798 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:21:05.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/1/new.json 2020-09-12T14:21:05.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 1 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/1 --osd-journal=td/osd-backfill-recovery-log/1/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQBR2Vxf3pLsDxAAUA8UEKp2wxNKvcUDIw9JyA== --osd-uuid 0c1e3a09-3808-4473-abe2-bae793ea6138 2020-09-12T14:21:05.834 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:05.833+0000 7fb3c8cbaec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:05.836 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:05.835+0000 7fb3c8cbaec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:05.838 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:05.836+0000 7fb3c8cbaec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:05.838 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:05.837+0000 7fb3c8cbaec0 -1 bluestore(td/osd-backfill-recovery-log/1/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/1/block: (2) No such file or directory 2020-09-12T14:21:05.839 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:05.837+0000 7fb3c8cbaec0 -1 bluestore(td/osd-backfill-recovery-log/1/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/1/block: (2) No such file or directory 2020-09-12T14:21:05.839 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:05.837+0000 7fb3c8cbaec0 -1 bluestore(td/osd-backfill-recovery-log/1/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/1/block: (2) No such file or directory 2020-09-12T14:21:05.940 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:05.939+0000 7fb3c8cbaec0 -1 bluestore(td/osd-backfill-recovery-log/1) _read_fsid unparsable uuid 2020-09-12T14:21:10.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/1/keyring 2020-09-12T14:21:10.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:21:10.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd1 key to auth repository 2020-09-12T14:21:10.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:21:10.526 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd1 key to auth repository 2020-09-12T14:21:11.091 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.1 2020-09-12T14:21:11.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 1 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/1 --osd-journal=td/osd-backfill-recovery-log/1/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:21:11.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:21:11.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:21:11.093 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T14:21:11.093 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:21:11.115 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:11.114+0000 7fea157f9ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:11.130 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:11.129+0000 7fea157f9ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:11.131 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:11.131+0000 7fea157f9ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:11.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 1 2020-09-12T14:21:11.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:21:11.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:21:11.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:21:11.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:21:11.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:21:11.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:21:11.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:21:11.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:21:11.495 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:21:11.647 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:11.646+0000 7fea157f9ec0 -1 Falling back to public interface 2020-09-12T14:21:11.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:21:12.507 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:12.507+0000 7fea157f9ec0 -1 osd.1 0 log_to_monitors {default=true} 2020-09-12T14:21:12.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:21:12.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:21:12.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:21:12.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:21:12.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:21:12.906 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:21:13.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:21:14.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:21:14.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:21:14.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 2 2020-09-12T14:21:14.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:21:14.319 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:21:14.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:21:14.730 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/113464,v1:127.0.0.1:6811/113464] [v2:127.0.0.1:6812/113464,v1:127.0.0.1:6813/113464] exists,up 0c1e3a09-3808-4473-abe2-bae793ea6138 2020-09-12T14:21:14.730 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:21:14.730 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:21:14.730 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:21:14.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:21:14.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 2 2020-09-12T14:21:14.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:21:14.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:21:14.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=2 2020-09-12T14:21:14.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:21:14.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/2 2020-09-12T14:21:14.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:21:14.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:21:14.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:21:14.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:21:14.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/2' 2020-09-12T14:21:14.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/2/journal' 2020-09-12T14:21:14.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:21:14.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2' 2020-09-12T14:21:14.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:21:14.733 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:21:14.733 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:21:14.733 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:21:14.733 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:21:14.733 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:21:14.734 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:21:14.734 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:21:14.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:21:14.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:21:14.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:21:14.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:21:14.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:21:14.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:21:14.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:21:14.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:21:14.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:21:14.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:21:14.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:21:14.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/2 2020-09-12T14:21:14.736 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:21:14.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=8de014d0-0eb4-41f1-8a47-c5ab2c5c8c9a 2020-09-12T14:21:14.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd2 8de014d0-0eb4-41f1-8a47-c5ab2c5c8c9a' 2020-09-12T14:21:14.736 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:21:14.737 INFO:tasks.workunit.client.0.smithi027.stdout:add osd2 8de014d0-0eb4-41f1-8a47-c5ab2c5c8c9a 2020-09-12T14:21:14.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQBa2VxfwFRzLBAAgPwYgu93hacoqJh5LaXvuA== 2020-09-12T14:21:14.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQBa2VxfwFRzLBAAgPwYgu93hacoqJh5LaXvuA=="}' 2020-09-12T14:21:14.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 8de014d0-0eb4-41f1-8a47-c5ab2c5c8c9a -i td/osd-backfill-recovery-log/2/new.json 2020-09-12T14:21:15.281 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:21:15.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/2/new.json 2020-09-12T14:21:15.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 2 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/2 --osd-journal=td/osd-backfill-recovery-log/2/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQBa2VxfwFRzLBAAgPwYgu93hacoqJh5LaXvuA== --osd-uuid 8de014d0-0eb4-41f1-8a47-c5ab2c5c8c9a 2020-09-12T14:21:15.318 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:15.318+0000 7fa1c97d2ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:15.320 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:15.320+0000 7fa1c97d2ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:15.321 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:15.321+0000 7fa1c97d2ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:15.322 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:15.321+0000 7fa1c97d2ec0 -1 bluestore(td/osd-backfill-recovery-log/2/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/2/block: (2) No such file or directory 2020-09-12T14:21:15.322 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:15.321+0000 7fa1c97d2ec0 -1 bluestore(td/osd-backfill-recovery-log/2/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/2/block: (2) No such file or directory 2020-09-12T14:21:15.322 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:15.321+0000 7fa1c97d2ec0 -1 bluestore(td/osd-backfill-recovery-log/2/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/2/block: (2) No such file or directory 2020-09-12T14:21:15.356 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:15.356+0000 7fa1c97d2ec0 -1 bluestore(td/osd-backfill-recovery-log/2) _read_fsid unparsable uuid 2020-09-12T14:21:20.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/2/keyring 2020-09-12T14:21:20.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:21:20.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd2 key to auth repository 2020-09-12T14:21:20.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:21:20.229 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd2 key to auth repository 2020-09-12T14:21:20.776 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.2 2020-09-12T14:21:20.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.2 2020-09-12T14:21:20.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 2 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/2 --osd-journal=td/osd-backfill-recovery-log/2/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:21:20.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:21:20.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:21:20.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:21:20.799 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:20.799+0000 7f8f049b4ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:20.831 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:20.830+0000 7f8f049b4ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:20.832 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:20.831+0000 7f8f049b4ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:21.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 2 2020-09-12T14:21:21.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:21:21.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T14:21:21.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:21:21.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:21:21.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:21:21.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:21:21.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:21:21.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:21:21.182 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:21:21.353 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:21.352+0000 7f8f049b4ec0 -1 Falling back to public interface 2020-09-12T14:21:21.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:21:22.207 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:22.207+0000 7f8f049b4ec0 -1 osd.2 0 log_to_monitors {default=true} 2020-09-12T14:21:22.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:21:22.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:21:22.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:21:22.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:21:22.590 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:21:22.590 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:21:23.043 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/114484,v1:127.0.0.1:6819/114484] [v2:127.0.0.1:6820/114484,v1:127.0.0.1:6821/114484] exists,up 8de014d0-0eb4-41f1-8a47-c5ab2c5c8c9a 2020-09-12T14:21:23.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:21:23.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:21:23.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:21:23.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:21:23.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 3 2020-09-12T14:21:23.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:21:23.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:21:23.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=3 2020-09-12T14:21:23.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:21:23.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/3 2020-09-12T14:21:23.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:21:23.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:21:23.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:21:23.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:21:23.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/3' 2020-09-12T14:21:23.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/3/journal' 2020-09-12T14:21:23.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:21:23.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2' 2020-09-12T14:21:23.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:21:23.046 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:21:23.046 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:21:23.046 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:21:23.047 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:21:23.047 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:21:23.047 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:21:23.047 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:21:23.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:21:23.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:21:23.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:21:23.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:21:23.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:21:23.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:21:23.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:21:23.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:21:23.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:21:23.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:21:23.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:21:23.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/3 2020-09-12T14:21:23.049 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:21:23.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=83ef93ba-86b2-4459-a86b-201764a70176 2020-09-12T14:21:23.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd3 83ef93ba-86b2-4459-a86b-201764a70176' 2020-09-12T14:21:23.050 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:21:23.050 INFO:tasks.workunit.client.0.smithi027.stdout:add osd3 83ef93ba-86b2-4459-a86b-201764a70176 2020-09-12T14:21:23.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQBj2VxfytJ+AxAAfnhOejP0gyTxkNwz8ZoL9w== 2020-09-12T14:21:23.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQBj2VxfytJ+AxAAfnhOejP0gyTxkNwz8ZoL9w=="}' 2020-09-12T14:21:23.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 83ef93ba-86b2-4459-a86b-201764a70176 -i td/osd-backfill-recovery-log/3/new.json 2020-09-12T14:21:23.740 INFO:tasks.workunit.client.0.smithi027.stdout:3 2020-09-12T14:21:23.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/3/new.json 2020-09-12T14:21:23.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 3 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/3 --osd-journal=td/osd-backfill-recovery-log/3/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQBj2VxfytJ+AxAAfnhOejP0gyTxkNwz8ZoL9w== --osd-uuid 83ef93ba-86b2-4459-a86b-201764a70176 2020-09-12T14:21:23.777 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:23.777+0000 7fa0fbdf8ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:23.779 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:23.779+0000 7fa0fbdf8ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:23.781 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:23.780+0000 7fa0fbdf8ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:23.781 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:23.781+0000 7fa0fbdf8ec0 -1 bluestore(td/osd-backfill-recovery-log/3/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/3/block: (2) No such file or directory 2020-09-12T14:21:23.782 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:23.781+0000 7fa0fbdf8ec0 -1 bluestore(td/osd-backfill-recovery-log/3/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/3/block: (2) No such file or directory 2020-09-12T14:21:23.782 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:23.781+0000 7fa0fbdf8ec0 -1 bluestore(td/osd-backfill-recovery-log/3/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/3/block: (2) No such file or directory 2020-09-12T14:21:23.874 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:23.874+0000 7fa0fbdf8ec0 -1 bluestore(td/osd-backfill-recovery-log/3) _read_fsid unparsable uuid 2020-09-12T14:21:28.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/3/keyring 2020-09-12T14:21:28.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:21:28.480 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd3 key to auth repository 2020-09-12T14:21:28.480 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:21:28.480 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd3 key to auth repository 2020-09-12T14:21:29.028 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.3 2020-09-12T14:21:29.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.3 2020-09-12T14:21:29.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:21:29.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 3 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/3 --osd-journal=td/osd-backfill-recovery-log/3/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:21:29.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:21:29.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:21:29.051 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:29.050+0000 7fcb0023aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:29.073 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:29.073+0000 7fcb0023aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:29.075 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:29.075+0000 7fcb0023aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:29.434 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 3 2020-09-12T14:21:29.434 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:21:29.434 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=3 2020-09-12T14:21:29.434 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:21:29.434 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:21:29.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:21:29.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:21:29.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:21:29.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.3 up' 2020-09-12T14:21:29.435 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:21:29.592 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:29.591+0000 7fcb0023aec0 -1 Falling back to public interface 2020-09-12T14:21:29.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:21:30.437 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:30.435+0000 7fcb0023aec0 -1 osd.3 0 log_to_monitors {default=true} 2020-09-12T14:21:30.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:21:30.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:21:30.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:21:30.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:21:30.835 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:21:30.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.3 up' 2020-09-12T14:21:31.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:21:32.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:21:32.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:21:32.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 2 2020-09-12T14:21:32.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:21:32.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.3 up' 2020-09-12T14:21:32.283 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:21:32.699 INFO:tasks.workunit.client.0.smithi027.stdout:osd.3 up in weight 1 up_from 20 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/115478,v1:127.0.0.1:6827/115478] [v2:127.0.0.1:6828/115478,v1:127.0.0.1:6829/115478] exists,up 83ef93ba-86b2-4459-a86b-201764a70176 2020-09-12T14:21:32.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:21:32.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:21:32.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:21:32.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:21:32.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 4 2020-09-12T14:21:32.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:21:32.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:21:32.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=4 2020-09-12T14:21:32.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:21:32.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/4 2020-09-12T14:21:32.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:21:32.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:21:32.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:21:32.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:21:32.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/4' 2020-09-12T14:21:32.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/4/journal' 2020-09-12T14:21:32.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:21:32.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2' 2020-09-12T14:21:32.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:21:32.702 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:21:32.703 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:21:32.703 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:21:32.703 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:21:32.703 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:21:32.703 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:21:32.703 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:21:32.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:21:32.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:21:32.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:21:32.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:21:32.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:21:32.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:21:32.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:21:32.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:21:32.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:21:32.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:21:32.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:21:32.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/4 2020-09-12T14:21:32.705 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:21:32.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=16335c4e-8f97-43e9-a805-4d5d932eca16 2020-09-12T14:21:32.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd4 16335c4e-8f97-43e9-a805-4d5d932eca16' 2020-09-12T14:21:32.706 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:21:32.706 INFO:tasks.workunit.client.0.smithi027.stdout:add osd4 16335c4e-8f97-43e9-a805-4d5d932eca16 2020-09-12T14:21:32.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQBs2VxffaqcKhAALHjTG7iaki7HTcRmDVT0Rw== 2020-09-12T14:21:32.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQBs2VxffaqcKhAALHjTG7iaki7HTcRmDVT0Rw=="}' 2020-09-12T14:21:32.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 16335c4e-8f97-43e9-a805-4d5d932eca16 -i td/osd-backfill-recovery-log/4/new.json 2020-09-12T14:21:33.322 INFO:tasks.workunit.client.0.smithi027.stdout:4 2020-09-12T14:21:33.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/4/new.json 2020-09-12T14:21:33.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 4 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/4 --osd-journal=td/osd-backfill-recovery-log/4/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQBs2VxffaqcKhAALHjTG7iaki7HTcRmDVT0Rw== --osd-uuid 16335c4e-8f97-43e9-a805-4d5d932eca16 2020-09-12T14:21:33.359 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:33.358+0000 7f5510d7fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:33.361 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:33.360+0000 7f5510d7fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:33.363 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:33.361+0000 7f5510d7fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:33.363 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:33.362+0000 7f5510d7fec0 -1 bluestore(td/osd-backfill-recovery-log/4/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/4/block: (2) No such file or directory 2020-09-12T14:21:33.365 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:33.362+0000 7f5510d7fec0 -1 bluestore(td/osd-backfill-recovery-log/4/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/4/block: (2) No such file or directory 2020-09-12T14:21:33.365 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:33.362+0000 7f5510d7fec0 -1 bluestore(td/osd-backfill-recovery-log/4/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/4/block: (2) No such file or directory 2020-09-12T14:21:33.486 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:33.485+0000 7f5510d7fec0 -1 bluestore(td/osd-backfill-recovery-log/4) _read_fsid unparsable uuid 2020-09-12T14:21:38.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/4/keyring 2020-09-12T14:21:38.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:21:38.413 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd4 key to auth repository 2020-09-12T14:21:38.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd4 key to auth repository 2020-09-12T14:21:38.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:21:38.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.4 2020-09-12T14:21:38.965 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.4 2020-09-12T14:21:38.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:21:38.966 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 4 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/4 --osd-journal=td/osd-backfill-recovery-log/4/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:21:38.966 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:21:38.966 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:21:38.988 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:38.987+0000 7f6408c63ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:38.990 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:38.989+0000 7f6408c63ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:38.991 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:38.990+0000 7f6408c63ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:39.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 4 2020-09-12T14:21:39.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:21:39.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=4 2020-09-12T14:21:39.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:21:39.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:21:39.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:21:39.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:21:39.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.4 up' 2020-09-12T14:21:39.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:21:39.400 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:21:39.507 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:39.506+0000 7f6408c63ec0 -1 Falling back to public interface 2020-09-12T14:21:39.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:21:40.346 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:40.344+0000 7f6408c63ec0 -1 osd.4 0 log_to_monitors {default=true} 2020-09-12T14:21:40.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:21:40.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:21:40.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:21:40.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:21:40.804 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:21:40.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.4 up' 2020-09-12T14:21:41.271 INFO:tasks.workunit.client.0.smithi027.stdout:osd.4 up in weight 1 up_from 25 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/116498,v1:127.0.0.1:6835/116498] [v2:127.0.0.1:6836/116498,v1:127.0.0.1:6837/116498] exists,up 16335c4e-8f97-43e9-a805-4d5d932eca16 2020-09-12T14:21:41.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:21:41.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:21:41.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:21:41.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:21:41.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 5 2020-09-12T14:21:41.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:21:41.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:21:41.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=5 2020-09-12T14:21:41.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:21:41.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/5 2020-09-12T14:21:41.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:21:41.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:21:41.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:21:41.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:21:41.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/5' 2020-09-12T14:21:41.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/5/journal' 2020-09-12T14:21:41.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:21:41.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2' 2020-09-12T14:21:41.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:21:41.274 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:21:41.274 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:21:41.275 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:21:41.275 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:21:41.275 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:21:41.275 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:21:41.275 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:21:41.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:21:41.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:21:41.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:21:41.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:21:41.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:21:41.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:21:41.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:21:41.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:21:41.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:21:41.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:21:41.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:21:41.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/5 2020-09-12T14:21:41.277 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:21:41.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=e41ca84f-c80b-41ef-8166-c88e9adfdfc2 2020-09-12T14:21:41.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd5 e41ca84f-c80b-41ef-8166-c88e9adfdfc2' 2020-09-12T14:21:41.278 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:21:41.278 INFO:tasks.workunit.client.0.smithi027.stdout:add osd5 e41ca84f-c80b-41ef-8166-c88e9adfdfc2 2020-09-12T14:21:41.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQB12Vxfn/gZERAAETlXXawiftPpN52aHRMT9A== 2020-09-12T14:21:41.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQB12Vxfn/gZERAAETlXXawiftPpN52aHRMT9A=="}' 2020-09-12T14:21:41.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new e41ca84f-c80b-41ef-8166-c88e9adfdfc2 -i td/osd-backfill-recovery-log/5/new.json 2020-09-12T14:21:41.953 INFO:tasks.workunit.client.0.smithi027.stdout:5 2020-09-12T14:21:41.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/5/new.json 2020-09-12T14:21:41.966 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 5 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/5 --osd-journal=td/osd-backfill-recovery-log/5/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQB12Vxfn/gZERAAETlXXawiftPpN52aHRMT9A== --osd-uuid e41ca84f-c80b-41ef-8166-c88e9adfdfc2 2020-09-12T14:21:41.991 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:41.989+0000 7fbc73580ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:41.992 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:41.991+0000 7fbc73580ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:41.994 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:41.992+0000 7fbc73580ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:41.994 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:41.993+0000 7fbc73580ec0 -1 bluestore(td/osd-backfill-recovery-log/5/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/5/block: (2) No such file or directory 2020-09-12T14:21:41.995 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:41.993+0000 7fbc73580ec0 -1 bluestore(td/osd-backfill-recovery-log/5/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/5/block: (2) No such file or directory 2020-09-12T14:21:41.995 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:41.993+0000 7fbc73580ec0 -1 bluestore(td/osd-backfill-recovery-log/5/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/5/block: (2) No such file or directory 2020-09-12T14:21:42.038 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:42.037+0000 7fbc73580ec0 -1 bluestore(td/osd-backfill-recovery-log/5) _read_fsid unparsable uuid 2020-09-12T14:21:46.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/5/keyring 2020-09-12T14:21:46.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:21:46.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd5 key to auth repository 2020-09-12T14:21:46.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:21:46.902 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd5 key to auth repository 2020-09-12T14:21:47.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.5 2020-09-12T14:21:47.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:21:47.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 5 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/5 --osd-journal=td/osd-backfill-recovery-log/5/journal --chdir= --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:21:47.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:21:47.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:21:47.443 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.5 2020-09-12T14:21:47.465 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:47.465+0000 7fa598437ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:47.467 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:47.466+0000 7fa598437ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:47.468 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:47.468+0000 7fa598437ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:21:47.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 5 2020-09-12T14:21:47.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:21:47.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=5 2020-09-12T14:21:47.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:21:47.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:21:47.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:21:47.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:21:47.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:21:47.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.5 up' 2020-09-12T14:21:47.847 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:21:47.994 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:47.993+0000 7fa598437ec0 -1 Falling back to public interface 2020-09-12T14:21:48.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:21:48.797 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:21:48.797+0000 7fa598437ec0 -1 osd.5 0 log_to_monitors {default=true} 2020-09-12T14:21:49.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:21:49.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:21:49.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:21:49.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:21:49.281 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:21:49.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.5 up' 2020-09-12T14:21:49.714 INFO:tasks.workunit.client.0.smithi027.stdout:osd.5 up in weight 1 up_from 30 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/117492,v1:127.0.0.1:6843/117492] [v2:127.0.0.1:6844/117492,v1:127.0.0.1:6845/117492] exists,up e41ca84f-c80b-41ef-8166-c88e9adfdfc2 2020-09-12T14:21:49.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:21:49.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:21:49.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:21:49.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:59: _common_test: create_pool test 1 1 2020-09-12T14:21:49.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test 1 1 2020-09-12T14:21:50.286 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test' already exists 2020-09-12T14:21:50.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:21:51.297 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: seq 1 150 2020-09-12T14:21:51.298 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:51.298 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-1 /etc/passwd 2020-09-12T14:21:51.677 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:51.678 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-2 /etc/passwd 2020-09-12T14:21:51.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:51.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-3 /etc/passwd 2020-09-12T14:21:52.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:52.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-4 /etc/passwd 2020-09-12T14:21:52.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:52.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-5 /etc/passwd 2020-09-12T14:21:52.598 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:52.598 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-6 /etc/passwd 2020-09-12T14:21:52.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:52.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-7 /etc/passwd 2020-09-12T14:21:52.908 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:52.908 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-8 /etc/passwd 2020-09-12T14:21:53.093 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:53.093 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-9 /etc/passwd 2020-09-12T14:21:53.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:53.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-10 /etc/passwd 2020-09-12T14:21:53.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:53.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-11 /etc/passwd 2020-09-12T14:21:53.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:53.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-12 /etc/passwd 2020-09-12T14:21:53.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:53.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-13 /etc/passwd 2020-09-12T14:21:53.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:53.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-14 /etc/passwd 2020-09-12T14:21:54.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:54.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-15 /etc/passwd 2020-09-12T14:21:54.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:54.240 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-16 /etc/passwd 2020-09-12T14:21:54.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:54.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-17 /etc/passwd 2020-09-12T14:21:54.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:54.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-18 /etc/passwd 2020-09-12T14:21:54.659 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:54.659 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-19 /etc/passwd 2020-09-12T14:21:54.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:54.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-20 /etc/passwd 2020-09-12T14:21:54.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:54.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-21 /etc/passwd 2020-09-12T14:21:55.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:55.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-22 /etc/passwd 2020-09-12T14:21:55.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:55.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-23 /etc/passwd 2020-09-12T14:21:55.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:55.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-24 /etc/passwd 2020-09-12T14:21:55.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:55.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-25 /etc/passwd 2020-09-12T14:21:55.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:55.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-26 /etc/passwd 2020-09-12T14:21:56.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:56.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-27 /etc/passwd 2020-09-12T14:21:56.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:56.151 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-28 /etc/passwd 2020-09-12T14:21:56.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:56.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-29 /etc/passwd 2020-09-12T14:21:56.477 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:56.477 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-30 /etc/passwd 2020-09-12T14:21:56.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:56.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-31 /etc/passwd 2020-09-12T14:21:56.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:56.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-32 /etc/passwd 2020-09-12T14:21:56.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:56.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-33 /etc/passwd 2020-09-12T14:21:57.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:57.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-34 /etc/passwd 2020-09-12T14:21:57.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:57.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-35 /etc/passwd 2020-09-12T14:21:57.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:57.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-36 /etc/passwd 2020-09-12T14:21:57.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:57.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-37 /etc/passwd 2020-09-12T14:21:57.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:57.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-38 /etc/passwd 2020-09-12T14:21:57.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:57.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-39 /etc/passwd 2020-09-12T14:21:58.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:58.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-40 /etc/passwd 2020-09-12T14:21:58.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:58.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-41 /etc/passwd 2020-09-12T14:21:58.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:58.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-42 /etc/passwd 2020-09-12T14:21:58.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:58.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-43 /etc/passwd 2020-09-12T14:21:58.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:58.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-44 /etc/passwd 2020-09-12T14:21:58.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:58.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-45 /etc/passwd 2020-09-12T14:21:59.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:59.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-46 /etc/passwd 2020-09-12T14:21:59.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:59.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-47 /etc/passwd 2020-09-12T14:21:59.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:59.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-48 /etc/passwd 2020-09-12T14:21:59.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:59.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-49 /etc/passwd 2020-09-12T14:21:59.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:59.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-50 /etc/passwd 2020-09-12T14:21:59.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:59.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-51 /etc/passwd 2020-09-12T14:21:59.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:21:59.997 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-52 /etc/passwd 2020-09-12T14:22:00.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:00.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-53 /etc/passwd 2020-09-12T14:22:00.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:00.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-54 /etc/passwd 2020-09-12T14:22:00.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:00.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-55 /etc/passwd 2020-09-12T14:22:00.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:00.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-56 /etc/passwd 2020-09-12T14:22:00.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:00.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-57 /etc/passwd 2020-09-12T14:22:00.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:00.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-58 /etc/passwd 2020-09-12T14:22:01.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:01.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-59 /etc/passwd 2020-09-12T14:22:01.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:01.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-60 /etc/passwd 2020-09-12T14:22:01.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:01.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-61 /etc/passwd 2020-09-12T14:22:01.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:01.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-62 /etc/passwd 2020-09-12T14:22:01.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:01.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-63 /etc/passwd 2020-09-12T14:22:02.083 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:02.083 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-64 /etc/passwd 2020-09-12T14:22:02.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:02.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-65 /etc/passwd 2020-09-12T14:22:02.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:02.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-66 /etc/passwd 2020-09-12T14:22:02.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:02.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-67 /etc/passwd 2020-09-12T14:22:02.669 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:02.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-68 /etc/passwd 2020-09-12T14:22:02.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:02.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-69 /etc/passwd 2020-09-12T14:22:02.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:02.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-70 /etc/passwd 2020-09-12T14:22:03.089 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:03.089 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-71 /etc/passwd 2020-09-12T14:22:03.231 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:03.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-72 /etc/passwd 2020-09-12T14:22:03.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:03.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-73 /etc/passwd 2020-09-12T14:22:03.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:03.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-74 /etc/passwd 2020-09-12T14:22:03.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:03.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-75 /etc/passwd 2020-09-12T14:22:03.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:03.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-76 /etc/passwd 2020-09-12T14:22:03.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:03.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-77 /etc/passwd 2020-09-12T14:22:04.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:04.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-78 /etc/passwd 2020-09-12T14:22:04.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:04.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-79 /etc/passwd 2020-09-12T14:22:04.431 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:04.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-80 /etc/passwd 2020-09-12T14:22:04.597 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:04.597 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-81 /etc/passwd 2020-09-12T14:22:04.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:04.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-82 /etc/passwd 2020-09-12T14:22:04.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:04.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-83 /etc/passwd 2020-09-12T14:22:05.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:05.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-84 /etc/passwd 2020-09-12T14:22:05.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:05.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-85 /etc/passwd 2020-09-12T14:22:05.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:05.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-86 /etc/passwd 2020-09-12T14:22:05.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:05.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-87 /etc/passwd 2020-09-12T14:22:05.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:05.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-88 /etc/passwd 2020-09-12T14:22:05.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:05.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-89 /etc/passwd 2020-09-12T14:22:05.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:05.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-90 /etc/passwd 2020-09-12T14:22:06.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:06.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-91 /etc/passwd 2020-09-12T14:22:06.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:06.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-92 /etc/passwd 2020-09-12T14:22:06.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:06.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-93 /etc/passwd 2020-09-12T14:22:06.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:06.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-94 /etc/passwd 2020-09-12T14:22:06.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:06.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-95 /etc/passwd 2020-09-12T14:22:06.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:06.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-96 /etc/passwd 2020-09-12T14:22:07.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:07.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-97 /etc/passwd 2020-09-12T14:22:07.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:07.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-98 /etc/passwd 2020-09-12T14:22:07.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:07.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-99 /etc/passwd 2020-09-12T14:22:07.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:07.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-100 /etc/passwd 2020-09-12T14:22:07.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:07.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-101 /etc/passwd 2020-09-12T14:22:07.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:07.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-102 /etc/passwd 2020-09-12T14:22:08.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:08.083 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-103 /etc/passwd 2020-09-12T14:22:08.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:08.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-104 /etc/passwd 2020-09-12T14:22:08.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:08.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-105 /etc/passwd 2020-09-12T14:22:08.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:08.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-106 /etc/passwd 2020-09-12T14:22:08.669 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:08.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-107 /etc/passwd 2020-09-12T14:22:08.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:08.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-108 /etc/passwd 2020-09-12T14:22:08.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:08.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-109 /etc/passwd 2020-09-12T14:22:09.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:09.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-110 /etc/passwd 2020-09-12T14:22:09.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:09.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-111 /etc/passwd 2020-09-12T14:22:09.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:09.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-112 /etc/passwd 2020-09-12T14:22:09.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:09.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-113 /etc/passwd 2020-09-12T14:22:09.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:09.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-114 /etc/passwd 2020-09-12T14:22:09.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:09.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-115 /etc/passwd 2020-09-12T14:22:10.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:10.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-116 /etc/passwd 2020-09-12T14:22:10.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:10.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-117 /etc/passwd 2020-09-12T14:22:10.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:10.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-118 /etc/passwd 2020-09-12T14:22:10.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:10.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-119 /etc/passwd 2020-09-12T14:22:10.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:10.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-120 /etc/passwd 2020-09-12T14:22:10.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:10.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-121 /etc/passwd 2020-09-12T14:22:11.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:11.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-122 /etc/passwd 2020-09-12T14:22:11.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:11.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-123 /etc/passwd 2020-09-12T14:22:11.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:11.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-124 /etc/passwd 2020-09-12T14:22:11.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:11.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-125 /etc/passwd 2020-09-12T14:22:11.761 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:11.761 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-126 /etc/passwd 2020-09-12T14:22:11.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:11.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-127 /etc/passwd 2020-09-12T14:22:12.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:12.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-128 /etc/passwd 2020-09-12T14:22:12.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:12.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-129 /etc/passwd 2020-09-12T14:22:12.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:12.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-130 /etc/passwd 2020-09-12T14:22:12.498 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:12.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-131 /etc/passwd 2020-09-12T14:22:12.671 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:12.671 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-132 /etc/passwd 2020-09-12T14:22:12.841 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:12.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-133 /etc/passwd 2020-09-12T14:22:12.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:12.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-134 /etc/passwd 2020-09-12T14:22:13.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:13.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-135 /etc/passwd 2020-09-12T14:22:13.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:13.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-136 /etc/passwd 2020-09-12T14:22:13.444 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:13.445 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-137 /etc/passwd 2020-09-12T14:22:13.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:13.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-138 /etc/passwd 2020-09-12T14:22:13.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:13.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-139 /etc/passwd 2020-09-12T14:22:13.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:13.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-140 /etc/passwd 2020-09-12T14:22:14.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:14.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-141 /etc/passwd 2020-09-12T14:22:14.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:14.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-142 /etc/passwd 2020-09-12T14:22:14.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:14.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-143 /etc/passwd 2020-09-12T14:22:14.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:14.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-144 /etc/passwd 2020-09-12T14:22:14.626 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:14.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-145 /etc/passwd 2020-09-12T14:22:14.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:14.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-146 /etc/passwd 2020-09-12T14:22:14.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:14.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-147 /etc/passwd 2020-09-12T14:22:15.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:15.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-148 /etc/passwd 2020-09-12T14:22:15.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:15.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-149 /etc/passwd 2020-09-12T14:22:15.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:22:15.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-150 /etc/passwd 2020-09-12T14:22:15.541 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:67: _common_test: ceph pg dump pgs --format=json 2020-09-12T14:22:15.541 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:67: _common_test: jq '.pg_stats[0].up[]' 2020-09-12T14:22:15.933 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:22:15.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:67: _common_test: ceph osd out 1 0 2 2020-09-12T14:22:16.569 INFO:tasks.workunit.client.0.smithi027.stderr:osd.1 is already out. osd.0 is already out. osd.2 is already out. 2020-09-12T14:22:16.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:68: _common_test: '[' 0 '!=' 0 ']' 2020-09-12T14:22:16.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:74: _common_test: sleep 1 2020-09-12T14:22:17.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:75: _common_test: wait_for_clean 2020-09-12T14:22:17.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:22:17.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:22:17.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:22:17.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:22:17.582 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 90 .1 2020-09-12T14:22:17.582 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:22:17.582 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:22:17.582 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:22:17.582 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:22:17.582 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:22:17.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:22:17.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:22:17.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:22:17.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:22:17.688 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:22:18.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:22:18.199 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:22:18.200 INFO:tasks.workunit.client.0.smithi027.stderr:2 2020-09-12T14:22:18.200 INFO:tasks.workunit.client.0.smithi027.stderr:3 2020-09-12T14:22:18.200 INFO:tasks.workunit.client.0.smithi027.stderr:4 2020-09-12T14:22:18.200 INFO:tasks.workunit.client.0.smithi027.stderr:5' 2020-09-12T14:22:18.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:22:18.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:22:18.201 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:22:18.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836497 2020-09-12T14:22:18.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836497 2020-09-12T14:22:18.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836497' 2020-09-12T14:22:18.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:22:18.416 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:22:18.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672975 2020-09-12T14:22:18.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672975 2020-09-12T14:22:18.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836497 1-42949672975' 2020-09-12T14:22:18.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:22:18.556 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:22:18.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509454 2020-09-12T14:22:18.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509454 2020-09-12T14:22:18.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836497 1-42949672975 2-64424509454' 2020-09-12T14:22:18.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:22:18.696 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2020-09-12T14:22:18.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=85899345932 2020-09-12T14:22:18.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 85899345932 2020-09-12T14:22:18.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836497 1-42949672975 2-64424509454 3-85899345932' 2020-09-12T14:22:18.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:22:18.835 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2020-09-12T14:22:18.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=107374182410 2020-09-12T14:22:18.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 107374182410 2020-09-12T14:22:18.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836497 1-42949672975 2-64424509454 3-85899345932 4-107374182410' 2020-09-12T14:22:18.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:22:18.976 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2020-09-12T14:22:19.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=128849018889 2020-09-12T14:22:19.116 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 128849018889 2020-09-12T14:22:19.116 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836497 1-42949672975 2-64424509454 3-85899345932 4-107374182410 5-128849018889' 2020-09-12T14:22:19.116 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:22:19.116 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836497 2020-09-12T14:22:19.117 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:22:19.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:22:19.117 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836497 2020-09-12T14:22:19.117 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:22:19.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836497 2020-09-12T14:22:19.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836497' 2020-09-12T14:22:19.118 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:22:19.119 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836497 2020-09-12T14:22:19.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836497 -lt 21474836497 2020-09-12T14:22:19.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:22:19.525 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672975 2020-09-12T14:22:19.525 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:22:19.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:22:19.525 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672975 2020-09-12T14:22:19.526 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:22:19.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672975 2020-09-12T14:22:19.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672975' 2020-09-12T14:22:19.527 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672975 2020-09-12T14:22:19.527 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:22:19.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672976 -lt 42949672975 2020-09-12T14:22:19.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:22:19.927 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509454 2020-09-12T14:22:19.927 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:22:19.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:22:19.928 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509454 2020-09-12T14:22:19.928 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:22:19.928 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509454 2020-09-12T14:22:19.929 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509454' 2020-09-12T14:22:19.929 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509454 2020-09-12T14:22:19.929 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:22:20.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509454 -lt 64424509454 2020-09-12T14:22:20.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:22:20.387 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 3-85899345932 2020-09-12T14:22:20.388 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:22:20.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=3 2020-09-12T14:22:20.388 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 3-85899345932 2020-09-12T14:22:20.389 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:22:20.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=85899345932 2020-09-12T14:22:20.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.3 seq 85899345932' 2020-09-12T14:22:20.390 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.3 seq 85899345932 2020-09-12T14:22:20.390 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 3 2020-09-12T14:22:20.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 85899345932 -lt 85899345932 2020-09-12T14:22:20.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:22:20.792 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 4-107374182410 2020-09-12T14:22:20.792 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:22:20.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=4 2020-09-12T14:22:20.793 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 4-107374182410 2020-09-12T14:22:20.793 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:22:20.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=107374182410 2020-09-12T14:22:20.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.4 seq 107374182410' 2020-09-12T14:22:20.794 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.4 seq 107374182410 2020-09-12T14:22:20.794 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 4 2020-09-12T14:22:21.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 107374182410 -lt 107374182410 2020-09-12T14:22:21.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:22:21.231 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 5-128849018889 2020-09-12T14:22:21.231 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:22:21.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=5 2020-09-12T14:22:21.232 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 5-128849018889 2020-09-12T14:22:21.232 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:22:21.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=128849018889 2020-09-12T14:22:21.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.5 seq 128849018889' 2020-09-12T14:22:21.234 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 5 2020-09-12T14:22:21.234 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.5 seq 128849018889 2020-09-12T14:22:21.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 128849018890 -lt 128849018889 2020-09-12T14:22:21.632 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:22:21.632 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:21.632 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:22.177 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 1 == 0 2020-09-12T14:22:22.177 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:22.177 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:22.177 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:22.178 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:22.178 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:22.178 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:22.178 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:22.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:22.569 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:22.569 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:22.570 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:23.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:23.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' -1 2020-09-12T14:22:23.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1598: wait_for_clean: loop=0 2020-09-12T14:22:23.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1599: wait_for_clean: num_active_clean=0 2020-09-12T14:22:23.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:23.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:23.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:23.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:23.205 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:23.205 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:23.205 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:23.205 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:23.205 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:23.205 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:23.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:23.597 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:23.597 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:23.597 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:24.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:24.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:24.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:24.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:24.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:24.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:24.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:24.134 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:24.134 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:24.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=5152 2020-09-12T14:22:24.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 5152 '!=' null 2020-09-12T14:22:24.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:24.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:24.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:24.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:24.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:24.788 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:24.788 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:24.789 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:24.789 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:24.789 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:24.789 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:25.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:25.184 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:25.184 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:25.185 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:25.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:25.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:25.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:25.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:25.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:25.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:25.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:25.773 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:25.774 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:26.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=4196 2020-09-12T14:22:26.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 4196 '!=' null 2020-09-12T14:22:26.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:26.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:26.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:26.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:26.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:26.429 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:26.429 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:26.430 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:26.430 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:26.430 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:26.430 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:26.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:26.818 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:26.819 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:26.819 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:27.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:27.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:27.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:27.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:27.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:27.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:27.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:27.347 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:27.347 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:27.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=17906 2020-09-12T14:22:27.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 17906 '!=' null 2020-09-12T14:22:27.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:27.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:27.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:28.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:28.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:28.003 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:28.003 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:28.004 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:28.004 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:28.004 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:28.004 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:28.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:28.394 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:28.395 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:28.395 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:28.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:28.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:28.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:28.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:28.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:28.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:28.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:28.926 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:28.927 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:29.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=15306 2020-09-12T14:22:29.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 15306 '!=' null 2020-09-12T14:22:29.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:29.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:29.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:29.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:29.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:29.561 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:29.561 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:29.561 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:29.562 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:29.562 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:29.562 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:29.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:29.950 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:29.950 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:29.951 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:30.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:30.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:30.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:30.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:30.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:30.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:30.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:30.520 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:30.520 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:31.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=15306 2020-09-12T14:22:31.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 15306 '!=' null 2020-09-12T14:22:31.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:31.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:31.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:31.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:31.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:31.150 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:31.150 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:31.151 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:31.151 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:31.151 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:31.151 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:31.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:31.545 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:31.545 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:31.546 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:32.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:32.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:32.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:32.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:32.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:32.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:32.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:32.186 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:32.186 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:32.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=25121 2020-09-12T14:22:32.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 25121 '!=' null 2020-09-12T14:22:32.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:32.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:32.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:32.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:32.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:32.819 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:32.819 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:32.820 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:32.820 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:32.820 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:32.820 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:33.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:33.208 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:33.208 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:33.209 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:33.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:33.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:33.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:33.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:33.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:33.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:33.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:33.756 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:33.756 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:34.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=22544 2020-09-12T14:22:34.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 22544 '!=' null 2020-09-12T14:22:34.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:34.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:34.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:34.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:34.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:34.402 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:34.402 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:34.403 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:34.403 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:34.403 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:34.403 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:34.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:34.792 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:34.793 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:34.793 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:35.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:35.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:35.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:35.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:35.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:35.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:35.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:35.322 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:35.322 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:35.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=22544 2020-09-12T14:22:35.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 22544 '!=' null 2020-09-12T14:22:35.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:35.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:35.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:36.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:36.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:36.007 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:36.007 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:36.008 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:36.008 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:36.008 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:36.008 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:36.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:36.399 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:36.399 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:36.399 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:36.934 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:36.934 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:36.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:36.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:36.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:36.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:36.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:36.935 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:36.935 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:37.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=32851 2020-09-12T14:22:37.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 32851 '!=' null 2020-09-12T14:22:37.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:37.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:37.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:37.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:37.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:37.592 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:37.592 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:37.592 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:37.592 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:37.592 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:37.593 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:37.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:37.982 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:37.982 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:37.982 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:38.617 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:38.617 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:38.617 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:38.617 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:38.617 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:38.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:38.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:38.618 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:38.618 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:39.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=21900 2020-09-12T14:22:39.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 21900 '!=' null 2020-09-12T14:22:39.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:39.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:39.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:39.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:39.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:39.256 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:39.256 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:39.257 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:39.257 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:39.257 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:39.257 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:39.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:39.649 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:39.649 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:39.649 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:40.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:40.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:40.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:40.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:40.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:40.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:40.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:40.185 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:40.185 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:40.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=21900 2020-09-12T14:22:40.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 21900 '!=' null 2020-09-12T14:22:40.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:40.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:40.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:40.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:40.871 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:40.871 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:40.871 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:40.872 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:40.872 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:40.872 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:40.872 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:41.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:41.267 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:41.267 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:41.268 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:41.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:41.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:41.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:41.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:41.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:41.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:41.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:41.800 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:41.800 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:42.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=32851 2020-09-12T14:22:42.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 32851 '!=' null 2020-09-12T14:22:42.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:42.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:42.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:42.457 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:42.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:42.458 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:42.458 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:42.458 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:42.458 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:42.459 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:42.459 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:42.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:42.850 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:42.851 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:42.851 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:43.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:43.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:43.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:43.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:43.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:43.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:43.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:43.383 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:43.384 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:43.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=21256 2020-09-12T14:22:43.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 21256 '!=' null 2020-09-12T14:22:43.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:43.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:43.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:44.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:44.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:44.018 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:44.019 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:44.019 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:44.019 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:44.019 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:44.019 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:44.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:44.419 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:44.420 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:44.420 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:44.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:44.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:44.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:44.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:44.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:44.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:44.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:44.948 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:44.948 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:45.498 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=21256 2020-09-12T14:22:45.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 21256 '!=' null 2020-09-12T14:22:45.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:45.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:45.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:45.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:45.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:45.600 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:45.600 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:45.600 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:45.601 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:45.601 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:45.601 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:45.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:45.991 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:45.991 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:45.991 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:46.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:46.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:46.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:46.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:46.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:46.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:46.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:46.566 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:46.566 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:47.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=21256 2020-09-12T14:22:47.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 21256 '!=' null 2020-09-12T14:22:47.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:47.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:47.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:47.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:47.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:47.245 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:47.245 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:47.245 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:47.245 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:47.246 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:47.246 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:47.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:47.639 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:47.640 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:47.640 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:48.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:48.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:48.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:48.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:48.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:48.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:48.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:48.169 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:48.170 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:48.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=32207 2020-09-12T14:22:48.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 32207 '!=' null 2020-09-12T14:22:48.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:48.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:48.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:48.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:48.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:48.855 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:48.855 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:48.856 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:48.856 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:48.856 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:48.856 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:49.246 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:49.247 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:49.247 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:49.247 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:49.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:49.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:49.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:49.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:49.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:49.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:49.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:49.784 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:49.784 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:50.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=21900 2020-09-12T14:22:50.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 21900 '!=' null 2020-09-12T14:22:50.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:50.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:50.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:50.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:50.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:50.475 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:50.475 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:50.475 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:50.475 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:50.475 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:50.476 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:50.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:50.868 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:50.869 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:50.869 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:51.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:51.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:51.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:51.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:51.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:51.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:51.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:51.398 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:51.398 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:51.928 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=33495 2020-09-12T14:22:51.928 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 33495 '!=' null 2020-09-12T14:22:51.929 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:51.929 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:51.929 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:52.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:52.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:52.030 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:52.030 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:52.030 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:52.031 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:52.031 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:52.031 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:52.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:52.421 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:52.422 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:52.422 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:52.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:52.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:52.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:52.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:52.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:52.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:52.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:52.948 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:52.948 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:53.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=22544 2020-09-12T14:22:53.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 22544 '!=' null 2020-09-12T14:22:53.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:53.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:53.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:53.612 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:53.612 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:53.612 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:53.612 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:53.613 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:53.613 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:53.613 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:53.613 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:54.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:54.006 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:54.006 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:54.007 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:54.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:54.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:54.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:54.547 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:54.547 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:54.547 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:54.547 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:54.547 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:54.547 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:55.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=22544 2020-09-12T14:22:55.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 22544 '!=' null 2020-09-12T14:22:55.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:55.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:55.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:55.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:55.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:55.304 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:55.304 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:55.304 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:55.305 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:55.305 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:55.305 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:55.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:55.697 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:55.697 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:55.697 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:56.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:56.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:56.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:56.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:56.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:56.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:56.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:56.226 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:56.226 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:56.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=22544 2020-09-12T14:22:56.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 22544 '!=' null 2020-09-12T14:22:56.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:56.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:56.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:56.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:56.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:56.891 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:56.891 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:56.891 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:56.891 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:56.892 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:56.892 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:57.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:57.292 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:57.293 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:57.293 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:57.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:57.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:57.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:57.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:57.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:57.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:57.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:57.828 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:57.828 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:58.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=33495 2020-09-12T14:22:58.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 33495 '!=' null 2020-09-12T14:22:58.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:58.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:58.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:22:58.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:22:58.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:22:58.465 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:22:58.465 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:22:58.465 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:22:58.465 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:22:58.466 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:22:58.466 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:22:58.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:22:58.854 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:22:58.854 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:22:58.854 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:22:59.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:22:59.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:22:59.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:22:59.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:22:59.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:22:59.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:22:59.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:22:59.395 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:22:59.395 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:22:59.934 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=22544 2020-09-12T14:22:59.934 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 22544 '!=' null 2020-09-12T14:22:59.934 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:22:59.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:22:59.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:23:00.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:23:00.035 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:23:00.035 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:23:00.035 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:23:00.035 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:23:00.035 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:23:00.036 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:23:00.036 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:23:00.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:23:00.429 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:23:00.430 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:23:00.430 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:23:00.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:23:00.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:23:00.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:23:00.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:23:00.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:23:00.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:23:00.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:23:00.958 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:23:00.958 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:23:01.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=34139 2020-09-12T14:23:01.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 34139 '!=' null 2020-09-12T14:23:01.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:23:01.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:23:01.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:23:01.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:23:01.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:23:01.638 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:23:01.638 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:23:01.638 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:23:01.639 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:23:01.639 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:23:01.639 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:23:02.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:23:02.058 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:23:02.058 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:23:02.059 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:23:02.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:23:02.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:23:02.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:23:02.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:23:02.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:23:02.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:23:02.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:23:02.588 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:23:02.588 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:23:03.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=34139 2020-09-12T14:23:03.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 34139 '!=' null 2020-09-12T14:23:03.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:23:03.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:23:03.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:23:03.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:23:03.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:23:03.227 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:23:03.227 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:23:03.228 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:23:03.228 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:23:03.228 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:23:03.228 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:23:03.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:23:03.619 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:23:03.619 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:23:03.619 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:23:04.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:23:04.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:23:04.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:23:04.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:23:04.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:23:04.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:23:04.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:23:04.282 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:23:04.282 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:23:04.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=22544 2020-09-12T14:23:04.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 22544 '!=' null 2020-09-12T14:23:04.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:23:04.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:23:04.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:23:04.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:23:04.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:23:04.932 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:23:04.932 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:23:04.932 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:23:04.932 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:23:04.932 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:23:04.932 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:23:05.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:23:05.329 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:23:05.330 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:23:05.330 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:23:05.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:23:05.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:23:05.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:23:05.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:23:05.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:23:05.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:23:05.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:23:05.858 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:23:05.859 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:23:06.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=17391 2020-09-12T14:23:06.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 17391 '!=' null 2020-09-12T14:23:06.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:23:06.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:23:06.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:23:06.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:23:06.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:23:06.489 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:23:06.489 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:23:06.490 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:23:06.490 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:23:06.490 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:23:06.490 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:23:06.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:23:06.879 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:23:06.880 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:23:06.880 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:23:07.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:23:07.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:23:07.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:23:07.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:23:07.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:23:07.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:23:07.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:23:07.441 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:23:07.442 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:23:07.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=25121 2020-09-12T14:23:07.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 25121 '!=' null 2020-09-12T14:23:07.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:23:07.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:23:07.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:23:08.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:23:08.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:23:08.073 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:23:08.073 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:23:08.074 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:23:08.074 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:23:08.074 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:23:08.074 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:23:08.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:23:08.467 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:23:08.468 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:23:08.468 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:23:08.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 1 2020-09-12T14:23:08.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T14:23:08.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T14:23:08.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:77: _common_test: flush_pg_stats 2020-09-12T14:23:08.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:23:08.996 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:23:09.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:23:09.402 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:23:09.402 INFO:tasks.workunit.client.0.smithi027.stderr:2 2020-09-12T14:23:09.402 INFO:tasks.workunit.client.0.smithi027.stderr:3 2020-09-12T14:23:09.402 INFO:tasks.workunit.client.0.smithi027.stderr:4 2020-09-12T14:23:09.402 INFO:tasks.workunit.client.0.smithi027.stderr:5' 2020-09-12T14:23:09.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:23:09.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:23:09.403 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:23:09.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836509 2020-09-12T14:23:09.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836509 2020-09-12T14:23:09.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836509' 2020-09-12T14:23:09.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:23:09.542 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:23:09.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672988 2020-09-12T14:23:09.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672988 2020-09-12T14:23:09.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836509 1-42949672988' 2020-09-12T14:23:09.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:23:09.682 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:23:09.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509466 2020-09-12T14:23:09.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509466 2020-09-12T14:23:09.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-64424509466' 2020-09-12T14:23:09.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:23:09.821 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2020-09-12T14:23:09.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=85899345944 2020-09-12T14:23:09.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 85899345944 2020-09-12T14:23:09.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-64424509466 3-85899345944' 2020-09-12T14:23:09.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:23:09.961 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2020-09-12T14:23:10.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=107374182422 2020-09-12T14:23:10.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 107374182422 2020-09-12T14:23:10.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-64424509466 3-85899345944 4-107374182422' 2020-09-12T14:23:10.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:23:10.102 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2020-09-12T14:23:10.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=128849018902 2020-09-12T14:23:10.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 128849018902 2020-09-12T14:23:10.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-64424509466 3-85899345944 4-107374182422 5-128849018902' 2020-09-12T14:23:10.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:23:10.243 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836509 2020-09-12T14:23:10.243 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:23:10.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:23:10.244 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836509 2020-09-12T14:23:10.244 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:23:10.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836509 2020-09-12T14:23:10.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836509' 2020-09-12T14:23:10.245 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:23:10.245 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836509 2020-09-12T14:23:10.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836507 -lt 21474836509 2020-09-12T14:23:10.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:23:11.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:23:11.647 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:23:12.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836510 -lt 21474836509 2020-09-12T14:23:12.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:23:12.122 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672988 2020-09-12T14:23:12.122 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:23:12.122 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:23:12.123 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672988 2020-09-12T14:23:12.123 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:23:12.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672988 2020-09-12T14:23:12.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672988' 2020-09-12T14:23:12.124 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:23:12.124 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672988 2020-09-12T14:23:12.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672988 -lt 42949672988 2020-09-12T14:23:12.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:23:12.525 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509466 2020-09-12T14:23:12.525 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:23:12.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:23:12.526 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509466 2020-09-12T14:23:12.526 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:23:12.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509466 2020-09-12T14:23:12.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509466' 2020-09-12T14:23:12.527 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509466 2020-09-12T14:23:12.528 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:23:12.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509466 -lt 64424509466 2020-09-12T14:23:12.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:23:12.969 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 3-85899345944 2020-09-12T14:23:12.969 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:23:12.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=3 2020-09-12T14:23:12.970 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 3-85899345944 2020-09-12T14:23:12.970 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:23:12.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=85899345944 2020-09-12T14:23:12.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.3 seq 85899345944' 2020-09-12T14:23:12.971 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 3 2020-09-12T14:23:12.972 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.3 seq 85899345944 2020-09-12T14:23:13.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 85899345945 -lt 85899345944 2020-09-12T14:23:13.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:23:13.370 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 4-107374182422 2020-09-12T14:23:13.370 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:23:13.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=4 2020-09-12T14:23:13.371 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 4-107374182422 2020-09-12T14:23:13.371 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:23:13.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=107374182422 2020-09-12T14:23:13.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.4 seq 107374182422' 2020-09-12T14:23:13.372 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.4 seq 107374182422 2020-09-12T14:23:13.372 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 4 2020-09-12T14:23:13.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 107374182423 -lt 107374182422 2020-09-12T14:23:13.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:23:13.781 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 5-128849018902 2020-09-12T14:23:13.782 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:23:13.782 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=5 2020-09-12T14:23:13.783 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 5-128849018902 2020-09-12T14:23:13.783 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:23:13.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=128849018902 2020-09-12T14:23:13.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.5 seq 128849018902' 2020-09-12T14:23:13.784 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.5 seq 128849018902 2020-09-12T14:23:13.784 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 5 2020-09-12T14:23:14.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 128849018902 -lt 128849018902 2020-09-12T14:23:14.206 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:79: _common_test: ceph pg dump pgs --format=json 2020-09-12T14:23:14.206 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:79: _common_test: jq '.pg_stats[0].up_primary' 2020-09-12T14:23:14.586 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:23:14.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:79: _common_test: newprimary=4 2020-09-12T14:23:14.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:80: _common_test: kill_daemons 2020-09-12T14:23:14.596 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:23:14.596 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:23:14.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:23:14.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:23:14.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:23:14.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:23:14.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:82: _common_test: ERRORS=0 2020-09-12T14:23:14.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:83: _common_test: _objectstore_tool_nodown td/osd-backfill-recovery-log 4 --no-mon-config --pgid 1.0 --op log 2020-09-12T14:23:14.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:83: _common_test: tee td/osd-backfill-recovery-log/result.log 2020-09-12T14:23:14.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1195: _objectstore_tool_nodown: local dir=td/osd-backfill-recovery-log 2020-09-12T14:23:14.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: _objectstore_tool_nodown: shift 2020-09-12T14:23:14.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: _objectstore_tool_nodown: local id=4 2020-09-12T14:23:14.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1198: _objectstore_tool_nodown: shift 2020-09-12T14:23:14.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: _objectstore_tool_nodown: local osd_data=td/osd-backfill-recovery-log/4 2020-09-12T14:23:14.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1202: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-backfill-recovery-log/4 --no-mon-config --pgid 1.0 --op log 2020-09-12T14:23:15.692 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:23:15.693 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_log_t": { 2020-09-12T14:23:15.693 INFO:tasks.workunit.client.0.smithi027.stdout: "head": "34'150", 2020-09-12T14:23:15.693 INFO:tasks.workunit.client.0.smithi027.stdout: "tail": "34'148", 2020-09-12T14:23:15.694 INFO:tasks.workunit.client.0.smithi027.stdout: "log": [ 2020-09-12T14:23:15.694 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.694 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:23:15.694 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:e9c179f9:::obj-149:head", 2020-09-12T14:23:15.695 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'149", 2020-09-12T14:23:15.695 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:23:15.695 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4635.0:1", 2020-09-12T14:23:15.695 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:23:15.695 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:22:15.265384+0000", 2020-09-12T14:23:15.695 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:23:15.695 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:23:15.696 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:23:15.696 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:23:15.696 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:23:15.696 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:23:15.697 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:23:15.697 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.697 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:23:15.697 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:23:15.697 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:23:15.697 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:23:15.697 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:23:15.698 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:23:15.698 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:23:15.698 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.698 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.698 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:23:15.698 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:cb39fd60:::obj-150:head", 2020-09-12T14:23:15.698 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'150", 2020-09-12T14:23:15.699 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:23:15.699 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4638.0:1", 2020-09-12T14:23:15.699 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:23:15.699 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:22:15.400425+0000", 2020-09-12T14:23:15.699 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:23:15.699 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:23:15.699 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:23:15.700 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:23:15.700 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:23:15.700 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:23:15.700 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:23:15.700 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.701 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:23:15.701 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:23:15.701 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:23:15.701 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:23:15.701 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:23:15.702 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:23:15.702 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:23:15.702 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:23:15.702 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:23:15.702 INFO:tasks.workunit.client.0.smithi027.stdout: "dups": [ 2020-09-12T14:23:15.702 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.703 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4191.0:1", 2020-09-12T14:23:15.703 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "33'1", 2020-09-12T14:23:15.703 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "1", 2020-09-12T14:23:15.703 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.703 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.704 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.704 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4194.0:1", 2020-09-12T14:23:15.704 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'2", 2020-09-12T14:23:15.704 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "2", 2020-09-12T14:23:15.704 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.704 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.705 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.705 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4197.0:1", 2020-09-12T14:23:15.705 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'3", 2020-09-12T14:23:15.705 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "3", 2020-09-12T14:23:15.705 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.705 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.705 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.706 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4200.0:1", 2020-09-12T14:23:15.706 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'4", 2020-09-12T14:23:15.706 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "4", 2020-09-12T14:23:15.706 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.706 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.706 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.706 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4203.0:1", 2020-09-12T14:23:15.707 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'5", 2020-09-12T14:23:15.707 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "5", 2020-09-12T14:23:15.707 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.707 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.707 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.707 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4206.0:1", 2020-09-12T14:23:15.707 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'6", 2020-09-12T14:23:15.708 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "6", 2020-09-12T14:23:15.708 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.708 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.708 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.708 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4209.0:1", 2020-09-12T14:23:15.708 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'7", 2020-09-12T14:23:15.708 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "7", 2020-09-12T14:23:15.709 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.709 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.709 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.709 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4212.0:1", 2020-09-12T14:23:15.709 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'8", 2020-09-12T14:23:15.709 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "8", 2020-09-12T14:23:15.709 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.709 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.710 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.710 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4215.0:1", 2020-09-12T14:23:15.710 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'9", 2020-09-12T14:23:15.710 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "9", 2020-09-12T14:23:15.711 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.711 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.711 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.711 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4218.0:1", 2020-09-12T14:23:15.711 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'10", 2020-09-12T14:23:15.711 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "10", 2020-09-12T14:23:15.712 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.712 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.712 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.712 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4221.0:1", 2020-09-12T14:23:15.712 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'11", 2020-09-12T14:23:15.713 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "11", 2020-09-12T14:23:15.713 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.713 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.713 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.713 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4224.0:1", 2020-09-12T14:23:15.713 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'12", 2020-09-12T14:23:15.713 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "12", 2020-09-12T14:23:15.714 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.714 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.714 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.714 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4227.0:1", 2020-09-12T14:23:15.714 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'13", 2020-09-12T14:23:15.714 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "13", 2020-09-12T14:23:15.714 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.714 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.715 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.715 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4230.0:1", 2020-09-12T14:23:15.715 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'14", 2020-09-12T14:23:15.715 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "14", 2020-09-12T14:23:15.715 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.715 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.715 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.716 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4233.0:1", 2020-09-12T14:23:15.716 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'15", 2020-09-12T14:23:15.716 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "15", 2020-09-12T14:23:15.716 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.716 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.716 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.716 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4236.0:1", 2020-09-12T14:23:15.717 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'16", 2020-09-12T14:23:15.717 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "16", 2020-09-12T14:23:15.717 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.717 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.717 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.717 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4239.0:1", 2020-09-12T14:23:15.718 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'17", 2020-09-12T14:23:15.718 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "17", 2020-09-12T14:23:15.718 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.718 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.718 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.718 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4242.0:1", 2020-09-12T14:23:15.718 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'18", 2020-09-12T14:23:15.719 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "18", 2020-09-12T14:23:15.719 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.719 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.719 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.719 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4245.0:1", 2020-09-12T14:23:15.719 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'19", 2020-09-12T14:23:15.720 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "19", 2020-09-12T14:23:15.720 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.720 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.720 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.720 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4248.0:1", 2020-09-12T14:23:15.720 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'20", 2020-09-12T14:23:15.721 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "20", 2020-09-12T14:23:15.721 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.721 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.721 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.721 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4251.0:1", 2020-09-12T14:23:15.721 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'21", 2020-09-12T14:23:15.722 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "21", 2020-09-12T14:23:15.722 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.722 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.722 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.722 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4254.0:1", 2020-09-12T14:23:15.722 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'22", 2020-09-12T14:23:15.723 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "22", 2020-09-12T14:23:15.723 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.723 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.723 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.723 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4257.0:1", 2020-09-12T14:23:15.723 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'23", 2020-09-12T14:23:15.724 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "23", 2020-09-12T14:23:15.724 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.724 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.724 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.724 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4260.0:1", 2020-09-12T14:23:15.724 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'24", 2020-09-12T14:23:15.725 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "24", 2020-09-12T14:23:15.725 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.725 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.725 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.725 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4263.0:1", 2020-09-12T14:23:15.725 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'25", 2020-09-12T14:23:15.725 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "25", 2020-09-12T14:23:15.726 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.726 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.726 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.726 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4266.0:1", 2020-09-12T14:23:15.726 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'26", 2020-09-12T14:23:15.726 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "26", 2020-09-12T14:23:15.726 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.726 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.727 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.727 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4269.0:1", 2020-09-12T14:23:15.727 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'27", 2020-09-12T14:23:15.727 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "27", 2020-09-12T14:23:15.727 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.727 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.727 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.728 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4272.0:1", 2020-09-12T14:23:15.728 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'28", 2020-09-12T14:23:15.728 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "28", 2020-09-12T14:23:15.728 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.728 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.729 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.729 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4275.0:1", 2020-09-12T14:23:15.729 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'29", 2020-09-12T14:23:15.729 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "29", 2020-09-12T14:23:15.729 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.729 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.729 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.730 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4278.0:1", 2020-09-12T14:23:15.730 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'30", 2020-09-12T14:23:15.730 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "30", 2020-09-12T14:23:15.730 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.730 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.730 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.730 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4281.0:1", 2020-09-12T14:23:15.731 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'31", 2020-09-12T14:23:15.731 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "31", 2020-09-12T14:23:15.731 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.731 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.731 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.731 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4284.0:1", 2020-09-12T14:23:15.731 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'32", 2020-09-12T14:23:15.731 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "32", 2020-09-12T14:23:15.732 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.732 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.732 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.732 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4287.0:1", 2020-09-12T14:23:15.732 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'33", 2020-09-12T14:23:15.732 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "33", 2020-09-12T14:23:15.732 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.733 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.733 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.733 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4290.0:1", 2020-09-12T14:23:15.733 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'34", 2020-09-12T14:23:15.733 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "34", 2020-09-12T14:23:15.734 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.734 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.734 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.734 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4293.0:1", 2020-09-12T14:23:15.734 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'35", 2020-09-12T14:23:15.734 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "35", 2020-09-12T14:23:15.735 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.735 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.735 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.735 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4296.0:1", 2020-09-12T14:23:15.735 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'36", 2020-09-12T14:23:15.735 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "36", 2020-09-12T14:23:15.736 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.736 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.736 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.736 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4299.0:1", 2020-09-12T14:23:15.736 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'37", 2020-09-12T14:23:15.736 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "37", 2020-09-12T14:23:15.736 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.737 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.737 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.737 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4302.0:1", 2020-09-12T14:23:15.737 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'38", 2020-09-12T14:23:15.737 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "38", 2020-09-12T14:23:15.737 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.737 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.737 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.738 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4305.0:1", 2020-09-12T14:23:15.738 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'39", 2020-09-12T14:23:15.738 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "39", 2020-09-12T14:23:15.738 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.738 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.738 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.738 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4308.0:1", 2020-09-12T14:23:15.738 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'40", 2020-09-12T14:23:15.739 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "40", 2020-09-12T14:23:15.739 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.739 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.739 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.739 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4311.0:1", 2020-09-12T14:23:15.739 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'41", 2020-09-12T14:23:15.739 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "41", 2020-09-12T14:23:15.740 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.740 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.740 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.740 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4314.0:1", 2020-09-12T14:23:15.740 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'42", 2020-09-12T14:23:15.740 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "42", 2020-09-12T14:23:15.740 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.740 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.741 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.741 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4317.0:1", 2020-09-12T14:23:15.741 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'43", 2020-09-12T14:23:15.741 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "43", 2020-09-12T14:23:15.741 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.741 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.741 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.741 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4320.0:1", 2020-09-12T14:23:15.742 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'44", 2020-09-12T14:23:15.742 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "44", 2020-09-12T14:23:15.742 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.742 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.742 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.742 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4323.0:1", 2020-09-12T14:23:15.743 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'45", 2020-09-12T14:23:15.743 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "45", 2020-09-12T14:23:15.743 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.743 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.743 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.743 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4326.0:1", 2020-09-12T14:23:15.744 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'46", 2020-09-12T14:23:15.744 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "46", 2020-09-12T14:23:15.744 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.744 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.744 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.744 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4329.0:1", 2020-09-12T14:23:15.744 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'47", 2020-09-12T14:23:15.745 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "47", 2020-09-12T14:23:15.745 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.745 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.745 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.745 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4332.0:1", 2020-09-12T14:23:15.745 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'48", 2020-09-12T14:23:15.745 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "48", 2020-09-12T14:23:15.746 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.746 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.746 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.746 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4335.0:1", 2020-09-12T14:23:15.746 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'49", 2020-09-12T14:23:15.746 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "49", 2020-09-12T14:23:15.746 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.746 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.747 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.747 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4338.0:1", 2020-09-12T14:23:15.747 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'50", 2020-09-12T14:23:15.747 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "50", 2020-09-12T14:23:15.747 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.747 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.747 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.748 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4341.0:1", 2020-09-12T14:23:15.748 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'51", 2020-09-12T14:23:15.748 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "51", 2020-09-12T14:23:15.748 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.748 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.749 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.749 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4344.0:1", 2020-09-12T14:23:15.749 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'52", 2020-09-12T14:23:15.749 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "52", 2020-09-12T14:23:15.749 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.750 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.750 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.750 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4347.0:1", 2020-09-12T14:23:15.750 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'53", 2020-09-12T14:23:15.750 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "53", 2020-09-12T14:23:15.750 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.750 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.751 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.751 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4350.0:1", 2020-09-12T14:23:15.751 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'54", 2020-09-12T14:23:15.751 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "54", 2020-09-12T14:23:15.751 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.751 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.751 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.751 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4353.0:1", 2020-09-12T14:23:15.752 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'55", 2020-09-12T14:23:15.752 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "55", 2020-09-12T14:23:15.752 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.752 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.752 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.752 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4356.0:1", 2020-09-12T14:23:15.752 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'56", 2020-09-12T14:23:15.753 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "56", 2020-09-12T14:23:15.753 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.753 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.753 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.753 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4359.0:1", 2020-09-12T14:23:15.753 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'57", 2020-09-12T14:23:15.753 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "57", 2020-09-12T14:23:15.753 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.754 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.754 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.754 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4362.0:1", 2020-09-12T14:23:15.754 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'58", 2020-09-12T14:23:15.755 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "58", 2020-09-12T14:23:15.755 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.755 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.755 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.755 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4365.0:1", 2020-09-12T14:23:15.755 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'59", 2020-09-12T14:23:15.755 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "59", 2020-09-12T14:23:15.755 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.756 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.756 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.756 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4368.0:1", 2020-09-12T14:23:15.756 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'60", 2020-09-12T14:23:15.756 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "60", 2020-09-12T14:23:15.756 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.756 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.757 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.757 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4371.0:1", 2020-09-12T14:23:15.757 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'61", 2020-09-12T14:23:15.757 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "61", 2020-09-12T14:23:15.757 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.757 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.757 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.758 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4374.0:1", 2020-09-12T14:23:15.758 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'62", 2020-09-12T14:23:15.758 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "62", 2020-09-12T14:23:15.758 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.758 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.758 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.758 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4377.0:1", 2020-09-12T14:23:15.759 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'63", 2020-09-12T14:23:15.759 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "63", 2020-09-12T14:23:15.759 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.759 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.759 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.759 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4380.0:1", 2020-09-12T14:23:15.759 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'64", 2020-09-12T14:23:15.759 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "64", 2020-09-12T14:23:15.760 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.760 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.760 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.760 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4383.0:1", 2020-09-12T14:23:15.760 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'65", 2020-09-12T14:23:15.760 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "65", 2020-09-12T14:23:15.760 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.761 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.761 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.761 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4386.0:1", 2020-09-12T14:23:15.761 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'66", 2020-09-12T14:23:15.761 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "66", 2020-09-12T14:23:15.761 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.761 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.761 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.762 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4389.0:1", 2020-09-12T14:23:15.762 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'67", 2020-09-12T14:23:15.762 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "67", 2020-09-12T14:23:15.762 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.762 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.762 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.762 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4392.0:1", 2020-09-12T14:23:15.762 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'68", 2020-09-12T14:23:15.763 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "68", 2020-09-12T14:23:15.763 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.763 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.763 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.763 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4395.0:1", 2020-09-12T14:23:15.763 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'69", 2020-09-12T14:23:15.763 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "69", 2020-09-12T14:23:15.763 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.764 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.764 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.764 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4398.0:1", 2020-09-12T14:23:15.764 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'70", 2020-09-12T14:23:15.764 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "70", 2020-09-12T14:23:15.764 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.764 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.765 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.765 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4401.0:1", 2020-09-12T14:23:15.765 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'71", 2020-09-12T14:23:15.765 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "71", 2020-09-12T14:23:15.765 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.765 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.766 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.766 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4404.0:1", 2020-09-12T14:23:15.766 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'72", 2020-09-12T14:23:15.766 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "72", 2020-09-12T14:23:15.766 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.766 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.766 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.767 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4407.0:1", 2020-09-12T14:23:15.767 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'73", 2020-09-12T14:23:15.767 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "73", 2020-09-12T14:23:15.767 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.767 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.767 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.767 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4410.0:1", 2020-09-12T14:23:15.768 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'74", 2020-09-12T14:23:15.768 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "74", 2020-09-12T14:23:15.768 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.768 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.768 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.768 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4413.0:1", 2020-09-12T14:23:15.769 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'75", 2020-09-12T14:23:15.769 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "75", 2020-09-12T14:23:15.769 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.769 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.769 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.769 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4416.0:1", 2020-09-12T14:23:15.769 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'76", 2020-09-12T14:23:15.770 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "76", 2020-09-12T14:23:15.770 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.770 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.770 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.770 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4419.0:1", 2020-09-12T14:23:15.770 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'77", 2020-09-12T14:23:15.770 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "77", 2020-09-12T14:23:15.771 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.771 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.771 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.771 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4422.0:1", 2020-09-12T14:23:15.771 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'78", 2020-09-12T14:23:15.771 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "78", 2020-09-12T14:23:15.771 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.772 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.772 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.772 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4425.0:1", 2020-09-12T14:23:15.772 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'79", 2020-09-12T14:23:15.772 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "79", 2020-09-12T14:23:15.772 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.773 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.773 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.773 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4428.0:1", 2020-09-12T14:23:15.773 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'80", 2020-09-12T14:23:15.773 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "80", 2020-09-12T14:23:15.773 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.773 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.774 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.774 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4431.0:1", 2020-09-12T14:23:15.774 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'81", 2020-09-12T14:23:15.774 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "81", 2020-09-12T14:23:15.774 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.775 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.775 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.775 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4434.0:1", 2020-09-12T14:23:15.775 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'82", 2020-09-12T14:23:15.775 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "82", 2020-09-12T14:23:15.776 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.776 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.776 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.776 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4437.0:1", 2020-09-12T14:23:15.776 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'83", 2020-09-12T14:23:15.776 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "83", 2020-09-12T14:23:15.776 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.776 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.777 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.777 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4440.0:1", 2020-09-12T14:23:15.777 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'84", 2020-09-12T14:23:15.777 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "84", 2020-09-12T14:23:15.777 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.777 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.777 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.778 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4443.0:1", 2020-09-12T14:23:15.778 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'85", 2020-09-12T14:23:15.778 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "85", 2020-09-12T14:23:15.778 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.778 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.778 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.778 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4446.0:1", 2020-09-12T14:23:15.779 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'86", 2020-09-12T14:23:15.779 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "86", 2020-09-12T14:23:15.779 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.779 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.779 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.779 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4449.0:1", 2020-09-12T14:23:15.779 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'87", 2020-09-12T14:23:15.779 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "87", 2020-09-12T14:23:15.780 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.780 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.780 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.780 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4452.0:1", 2020-09-12T14:23:15.780 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'88", 2020-09-12T14:23:15.780 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "88", 2020-09-12T14:23:15.780 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.780 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.781 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.781 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4455.0:1", 2020-09-12T14:23:15.781 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'89", 2020-09-12T14:23:15.781 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "89", 2020-09-12T14:23:15.781 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.781 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.781 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.782 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4458.0:1", 2020-09-12T14:23:15.782 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'90", 2020-09-12T14:23:15.782 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "90", 2020-09-12T14:23:15.782 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.782 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.782 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.783 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4461.0:1", 2020-09-12T14:23:15.783 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'91", 2020-09-12T14:23:15.783 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "91", 2020-09-12T14:23:15.783 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.783 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.783 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.783 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4464.0:1", 2020-09-12T14:23:15.783 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'92", 2020-09-12T14:23:15.784 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "92", 2020-09-12T14:23:15.784 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.784 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.784 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.784 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4467.0:1", 2020-09-12T14:23:15.784 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'93", 2020-09-12T14:23:15.784 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "93", 2020-09-12T14:23:15.784 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.785 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.785 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.785 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4470.0:1", 2020-09-12T14:23:15.785 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'94", 2020-09-12T14:23:15.785 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "94", 2020-09-12T14:23:15.785 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.785 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.786 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.786 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4473.0:1", 2020-09-12T14:23:15.786 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'95", 2020-09-12T14:23:15.786 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "95", 2020-09-12T14:23:15.786 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.786 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.786 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.786 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4476.0:1", 2020-09-12T14:23:15.787 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'96", 2020-09-12T14:23:15.787 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "96", 2020-09-12T14:23:15.787 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.787 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.787 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.787 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4479.0:1", 2020-09-12T14:23:15.787 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'97", 2020-09-12T14:23:15.787 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "97", 2020-09-12T14:23:15.788 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.788 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.788 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.788 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4482.0:1", 2020-09-12T14:23:15.788 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'98", 2020-09-12T14:23:15.788 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "98", 2020-09-12T14:23:15.788 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.788 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.789 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.789 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4485.0:1", 2020-09-12T14:23:15.789 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'99", 2020-09-12T14:23:15.789 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "99", 2020-09-12T14:23:15.789 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.789 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.789 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.789 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4488.0:1", 2020-09-12T14:23:15.790 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'100", 2020-09-12T14:23:15.790 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "100", 2020-09-12T14:23:15.790 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.790 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.790 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.790 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4491.0:1", 2020-09-12T14:23:15.790 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'101", 2020-09-12T14:23:15.791 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "101", 2020-09-12T14:23:15.791 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.791 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.791 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.791 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4494.0:1", 2020-09-12T14:23:15.791 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'102", 2020-09-12T14:23:15.791 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "102", 2020-09-12T14:23:15.791 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.792 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.792 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.792 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4497.0:1", 2020-09-12T14:23:15.792 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'103", 2020-09-12T14:23:15.792 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "103", 2020-09-12T14:23:15.792 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.792 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.792 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.793 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4500.0:1", 2020-09-12T14:23:15.793 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'104", 2020-09-12T14:23:15.793 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "104", 2020-09-12T14:23:15.793 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.793 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.793 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.793 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4503.0:1", 2020-09-12T14:23:15.793 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'105", 2020-09-12T14:23:15.794 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "105", 2020-09-12T14:23:15.794 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.794 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.794 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.794 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4506.0:1", 2020-09-12T14:23:15.794 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'106", 2020-09-12T14:23:15.794 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "106", 2020-09-12T14:23:15.794 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.795 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.795 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.795 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4509.0:1", 2020-09-12T14:23:15.795 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'107", 2020-09-12T14:23:15.795 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "107", 2020-09-12T14:23:15.795 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.795 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.796 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.796 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4512.0:1", 2020-09-12T14:23:15.796 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'108", 2020-09-12T14:23:15.796 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "108", 2020-09-12T14:23:15.796 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.796 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.796 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.796 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4515.0:1", 2020-09-12T14:23:15.797 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'109", 2020-09-12T14:23:15.797 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "109", 2020-09-12T14:23:15.797 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.797 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.797 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.797 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4518.0:1", 2020-09-12T14:23:15.797 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'110", 2020-09-12T14:23:15.797 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "110", 2020-09-12T14:23:15.798 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.798 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.798 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.798 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4521.0:1", 2020-09-12T14:23:15.798 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'111", 2020-09-12T14:23:15.798 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "111", 2020-09-12T14:23:15.798 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.798 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.799 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.799 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4524.0:1", 2020-09-12T14:23:15.799 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'112", 2020-09-12T14:23:15.799 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "112", 2020-09-12T14:23:15.799 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.799 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.799 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.799 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4527.0:1", 2020-09-12T14:23:15.800 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'113", 2020-09-12T14:23:15.800 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "113", 2020-09-12T14:23:15.800 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.800 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.800 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.800 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4530.0:1", 2020-09-12T14:23:15.800 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'114", 2020-09-12T14:23:15.800 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "114", 2020-09-12T14:23:15.801 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.801 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.801 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.801 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4533.0:1", 2020-09-12T14:23:15.801 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'115", 2020-09-12T14:23:15.801 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "115", 2020-09-12T14:23:15.801 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.801 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.802 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.802 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4536.0:1", 2020-09-12T14:23:15.802 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'116", 2020-09-12T14:23:15.802 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "116", 2020-09-12T14:23:15.802 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.802 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.802 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.803 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4539.0:1", 2020-09-12T14:23:15.803 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'117", 2020-09-12T14:23:15.803 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "117", 2020-09-12T14:23:15.803 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.803 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.803 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.803 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4542.0:1", 2020-09-12T14:23:15.803 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'118", 2020-09-12T14:23:15.804 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "118", 2020-09-12T14:23:15.804 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.804 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.804 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.804 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4545.0:1", 2020-09-12T14:23:15.804 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'119", 2020-09-12T14:23:15.804 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "119", 2020-09-12T14:23:15.804 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.805 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.805 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.805 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4548.0:1", 2020-09-12T14:23:15.805 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'120", 2020-09-12T14:23:15.805 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "120", 2020-09-12T14:23:15.805 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.805 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.805 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.806 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4551.0:1", 2020-09-12T14:23:15.806 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'121", 2020-09-12T14:23:15.806 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "121", 2020-09-12T14:23:15.806 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.806 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.806 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.806 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4554.0:1", 2020-09-12T14:23:15.807 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'122", 2020-09-12T14:23:15.807 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "122", 2020-09-12T14:23:15.807 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.807 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.807 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.807 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4557.0:1", 2020-09-12T14:23:15.807 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'123", 2020-09-12T14:23:15.807 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "123", 2020-09-12T14:23:15.808 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.808 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.808 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.808 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4560.0:1", 2020-09-12T14:23:15.808 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'124", 2020-09-12T14:23:15.808 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "124", 2020-09-12T14:23:15.808 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.808 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.809 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.809 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4563.0:1", 2020-09-12T14:23:15.809 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'125", 2020-09-12T14:23:15.809 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "125", 2020-09-12T14:23:15.809 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.809 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.809 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.809 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4566.0:1", 2020-09-12T14:23:15.810 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'126", 2020-09-12T14:23:15.810 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "126", 2020-09-12T14:23:15.810 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.810 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.810 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.810 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4569.0:1", 2020-09-12T14:23:15.810 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'127", 2020-09-12T14:23:15.811 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "127", 2020-09-12T14:23:15.811 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.811 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.811 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.811 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4572.0:1", 2020-09-12T14:23:15.811 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'128", 2020-09-12T14:23:15.811 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "128", 2020-09-12T14:23:15.811 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.812 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.812 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.812 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4575.0:1", 2020-09-12T14:23:15.812 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'129", 2020-09-12T14:23:15.812 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "129", 2020-09-12T14:23:15.812 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.812 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.812 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.813 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4578.0:1", 2020-09-12T14:23:15.813 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'130", 2020-09-12T14:23:15.813 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "130", 2020-09-12T14:23:15.813 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.813 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.813 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.813 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4581.0:1", 2020-09-12T14:23:15.813 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'131", 2020-09-12T14:23:15.814 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "131", 2020-09-12T14:23:15.814 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.814 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.814 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.814 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4584.0:1", 2020-09-12T14:23:15.814 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'132", 2020-09-12T14:23:15.814 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "132", 2020-09-12T14:23:15.814 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.815 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.815 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.815 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4587.0:1", 2020-09-12T14:23:15.815 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'133", 2020-09-12T14:23:15.815 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "133", 2020-09-12T14:23:15.815 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.815 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.815 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.816 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4590.0:1", 2020-09-12T14:23:15.816 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'134", 2020-09-12T14:23:15.816 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "134", 2020-09-12T14:23:15.816 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.816 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.816 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.816 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4593.0:1", 2020-09-12T14:23:15.816 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'135", 2020-09-12T14:23:15.817 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "135", 2020-09-12T14:23:15.817 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.817 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.817 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.817 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4596.0:1", 2020-09-12T14:23:15.817 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'136", 2020-09-12T14:23:15.817 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "136", 2020-09-12T14:23:15.818 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.818 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.818 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.818 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4599.0:1", 2020-09-12T14:23:15.818 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'137", 2020-09-12T14:23:15.818 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "137", 2020-09-12T14:23:15.818 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.818 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.819 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.819 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4602.0:1", 2020-09-12T14:23:15.819 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'138", 2020-09-12T14:23:15.819 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "138", 2020-09-12T14:23:15.819 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.819 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.819 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.819 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4605.0:1", 2020-09-12T14:23:15.820 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'139", 2020-09-12T14:23:15.820 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "139", 2020-09-12T14:23:15.820 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.820 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.820 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.820 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4608.0:1", 2020-09-12T14:23:15.820 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'140", 2020-09-12T14:23:15.820 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "140", 2020-09-12T14:23:15.821 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.821 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.821 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.821 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4611.0:1", 2020-09-12T14:23:15.821 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'141", 2020-09-12T14:23:15.821 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "141", 2020-09-12T14:23:15.821 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.821 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.822 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.822 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4614.0:1", 2020-09-12T14:23:15.822 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'142", 2020-09-12T14:23:15.822 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "142", 2020-09-12T14:23:15.822 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.822 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.822 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.822 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4617.0:1", 2020-09-12T14:23:15.823 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'143", 2020-09-12T14:23:15.823 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "143", 2020-09-12T14:23:15.823 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.823 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.823 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.823 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4620.0:1", 2020-09-12T14:23:15.823 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'144", 2020-09-12T14:23:15.824 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "144", 2020-09-12T14:23:15.824 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.824 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.824 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.824 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4623.0:1", 2020-09-12T14:23:15.824 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'145", 2020-09-12T14:23:15.824 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "145", 2020-09-12T14:23:15.824 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.825 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.825 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.825 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4626.0:1", 2020-09-12T14:23:15.825 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'146", 2020-09-12T14:23:15.825 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "146", 2020-09-12T14:23:15.825 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.825 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.825 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.826 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4629.0:1", 2020-09-12T14:23:15.826 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'147", 2020-09-12T14:23:15.955 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "147", 2020-09-12T14:23:15.956 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.956 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.956 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:23:15.957 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4632.0:1", 2020-09-12T14:23:15.957 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'148", 2020-09-12T14:23:15.957 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "148", 2020-09-12T14:23:15.957 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:23:15.958 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:23:15.958 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:23:15.958 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:23:15.958 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_missing_t": { 2020-09-12T14:23:15.958 INFO:tasks.workunit.client.0.smithi027.stdout: "missing": [], 2020-09-12T14:23:15.959 INFO:tasks.workunit.client.0.smithi027.stdout: "may_include_deletes": true 2020-09-12T14:23:15.959 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:23:15.959 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:23:15.960 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:84: _common_test: jq '.pg_log_t.log | length' td/osd-backfill-recovery-log/result.log 2020-09-12T14:23:15.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:84: _common_test: LOGLEN=2 2020-09-12T14:23:15.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:85: _common_test: '[' 2 '!=' 2 ']' 2020-09-12T14:23:15.961 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:89: _common_test: jq '.pg_log_t.dups | length' td/osd-backfill-recovery-log/result.log 2020-09-12T14:23:15.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:89: _common_test: DUPSLEN=148 2020-09-12T14:23:15.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:90: _common_test: '[' 148 '!=' 148 ']' 2020-09-12T14:23:15.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:94: _common_test: grep 'copy_up_to\|copy_after' td/osd-backfill-recovery-log/osd.0.log td/osd-backfill-recovery-log/osd.1.log td/osd-backfill-recovery-log/osd.2.log td/osd-backfill-recovery-log/osd.3.log td/osd-backfill-recovery-log/osd.4.log td/osd-backfill-recovery-log/osd.5.log 2020-09-12T14:23:15.966 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to max 2 2020-09-12T14:23:15.967 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to copy log version 34'150 2020-09-12T14:23:15.967 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to copy log version 34'149 2020-09-12T14:23:15.967 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after earliest_dup_version 0 2020-09-12T14:23:15.967 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 33'1 2020-09-12T14:23:15.968 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'2 2020-09-12T14:23:15.968 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'3 2020-09-12T14:23:15.968 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'4 2020-09-12T14:23:15.968 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'5 2020-09-12T14:23:15.968 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'6 2020-09-12T14:23:15.968 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'7 2020-09-12T14:23:15.969 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'8 2020-09-12T14:23:15.969 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'9 2020-09-12T14:23:15.969 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'10 2020-09-12T14:23:15.969 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'11 2020-09-12T14:23:15.969 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'12 2020-09-12T14:23:15.970 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'13 2020-09-12T14:23:15.970 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'14 2020-09-12T14:23:15.970 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'15 2020-09-12T14:23:15.970 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'16 2020-09-12T14:23:15.970 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'17 2020-09-12T14:23:15.971 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'18 2020-09-12T14:23:15.971 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'19 2020-09-12T14:23:15.971 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'20 2020-09-12T14:23:15.971 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'21 2020-09-12T14:23:15.971 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'22 2020-09-12T14:23:15.971 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'23 2020-09-12T14:23:15.972 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'24 2020-09-12T14:23:15.972 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'25 2020-09-12T14:23:15.972 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'26 2020-09-12T14:23:15.972 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'27 2020-09-12T14:23:15.972 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'28 2020-09-12T14:23:15.973 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'29 2020-09-12T14:23:15.973 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'30 2020-09-12T14:23:15.973 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'31 2020-09-12T14:23:15.973 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'32 2020-09-12T14:23:15.973 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'33 2020-09-12T14:23:15.973 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'34 2020-09-12T14:23:15.973 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'35 2020-09-12T14:23:15.974 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'36 2020-09-12T14:23:15.974 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'37 2020-09-12T14:23:15.974 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'38 2020-09-12T14:23:15.974 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'39 2020-09-12T14:23:15.974 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'40 2020-09-12T14:23:15.974 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'41 2020-09-12T14:23:15.974 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'42 2020-09-12T14:23:15.974 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'43 2020-09-12T14:23:15.975 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'44 2020-09-12T14:23:15.975 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'45 2020-09-12T14:23:15.975 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'46 2020-09-12T14:23:15.975 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'47 2020-09-12T14:23:15.975 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'48 2020-09-12T14:23:15.975 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'49 2020-09-12T14:23:15.975 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'50 2020-09-12T14:23:15.976 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'51 2020-09-12T14:23:15.976 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'52 2020-09-12T14:23:15.976 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'53 2020-09-12T14:23:15.976 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'54 2020-09-12T14:23:15.976 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'55 2020-09-12T14:23:15.976 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'56 2020-09-12T14:23:15.976 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'57 2020-09-12T14:23:15.976 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'58 2020-09-12T14:23:15.977 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'59 2020-09-12T14:23:15.977 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'60 2020-09-12T14:23:15.977 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'61 2020-09-12T14:23:15.977 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'62 2020-09-12T14:23:15.977 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'63 2020-09-12T14:23:15.977 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'64 2020-09-12T14:23:15.978 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'65 2020-09-12T14:23:15.978 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'66 2020-09-12T14:23:15.978 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'67 2020-09-12T14:23:15.978 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'68 2020-09-12T14:23:15.978 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'69 2020-09-12T14:23:15.978 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'70 2020-09-12T14:23:15.979 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'71 2020-09-12T14:23:15.979 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'72 2020-09-12T14:23:15.979 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'73 2020-09-12T14:23:15.979 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'74 2020-09-12T14:23:15.979 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'75 2020-09-12T14:23:15.979 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'76 2020-09-12T14:23:15.979 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'77 2020-09-12T14:23:15.979 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'78 2020-09-12T14:23:15.980 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'79 2020-09-12T14:23:15.980 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'80 2020-09-12T14:23:15.980 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'81 2020-09-12T14:23:15.980 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'82 2020-09-12T14:23:15.980 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'83 2020-09-12T14:23:15.980 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'84 2020-09-12T14:23:15.980 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'85 2020-09-12T14:23:15.981 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'86 2020-09-12T14:23:15.981 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'87 2020-09-12T14:23:15.981 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'88 2020-09-12T14:23:15.981 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'89 2020-09-12T14:23:15.981 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'90 2020-09-12T14:23:15.981 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'91 2020-09-12T14:23:15.982 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'92 2020-09-12T14:23:15.982 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'93 2020-09-12T14:23:15.982 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'94 2020-09-12T14:23:15.982 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'95 2020-09-12T14:23:15.982 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'96 2020-09-12T14:23:15.982 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'97 2020-09-12T14:23:15.982 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'98 2020-09-12T14:23:15.983 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'99 2020-09-12T14:23:15.983 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'100 2020-09-12T14:23:15.983 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'101 2020-09-12T14:23:15.983 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'102 2020-09-12T14:23:15.983 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'103 2020-09-12T14:23:15.984 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'104 2020-09-12T14:23:15.984 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'105 2020-09-12T14:23:15.984 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'106 2020-09-12T14:23:15.984 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'107 2020-09-12T14:23:15.984 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'108 2020-09-12T14:23:15.984 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'109 2020-09-12T14:23:15.984 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'110 2020-09-12T14:23:15.985 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'111 2020-09-12T14:23:15.985 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'112 2020-09-12T14:23:15.985 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'113 2020-09-12T14:23:15.985 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'114 2020-09-12T14:23:15.985 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'115 2020-09-12T14:23:15.985 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'116 2020-09-12T14:23:15.985 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'117 2020-09-12T14:23:15.986 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'118 2020-09-12T14:23:15.986 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'119 2020-09-12T14:23:15.986 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'120 2020-09-12T14:23:15.986 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'121 2020-09-12T14:23:15.986 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'122 2020-09-12T14:23:15.986 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'123 2020-09-12T14:23:15.987 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'124 2020-09-12T14:23:15.987 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'125 2020-09-12T14:23:15.987 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'126 2020-09-12T14:23:15.987 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'127 2020-09-12T14:23:15.987 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'128 2020-09-12T14:23:15.988 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'129 2020-09-12T14:23:15.988 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'130 2020-09-12T14:23:15.988 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'131 2020-09-12T14:23:15.988 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'132 2020-09-12T14:23:15.989 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'133 2020-09-12T14:23:15.989 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'134 2020-09-12T14:23:15.989 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'135 2020-09-12T14:23:15.989 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'136 2020-09-12T14:23:15.989 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'137 2020-09-12T14:23:15.989 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'138 2020-09-12T14:23:15.989 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'139 2020-09-12T14:23:15.990 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'140 2020-09-12T14:23:15.990 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'141 2020-09-12T14:23:15.990 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'142 2020-09-12T14:23:15.990 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'143 2020-09-12T14:23:15.990 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'144 2020-09-12T14:23:15.990 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'145 2020-09-12T14:23:15.990 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'146 2020-09-12T14:23:15.991 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'147 2020-09-12T14:23:15.991 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'148 2020-09-12T14:23:15.991 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to max 2 2020-09-12T14:23:15.991 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to copy log version 34'150 2020-09-12T14:23:15.991 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to copy log version 34'149 2020-09-12T14:23:15.991 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after earliest_dup_version 0 2020-09-12T14:23:15.991 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 33'1 2020-09-12T14:23:15.992 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'2 2020-09-12T14:23:15.992 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'3 2020-09-12T14:23:15.992 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'4 2020-09-12T14:23:15.992 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'5 2020-09-12T14:23:15.992 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'6 2020-09-12T14:23:15.992 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'7 2020-09-12T14:23:15.993 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'8 2020-09-12T14:23:15.993 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'9 2020-09-12T14:23:15.993 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'10 2020-09-12T14:23:15.993 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'11 2020-09-12T14:23:15.993 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'12 2020-09-12T14:23:15.993 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'13 2020-09-12T14:23:15.993 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'14 2020-09-12T14:23:15.994 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'15 2020-09-12T14:23:15.994 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'16 2020-09-12T14:23:15.994 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'17 2020-09-12T14:23:15.994 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'18 2020-09-12T14:23:15.994 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'19 2020-09-12T14:23:15.994 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'20 2020-09-12T14:23:15.994 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'21 2020-09-12T14:23:15.995 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'22 2020-09-12T14:23:15.995 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'23 2020-09-12T14:23:15.995 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'24 2020-09-12T14:23:15.995 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'25 2020-09-12T14:23:15.995 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'26 2020-09-12T14:23:15.995 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'27 2020-09-12T14:23:15.995 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'28 2020-09-12T14:23:15.995 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'29 2020-09-12T14:23:15.996 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'30 2020-09-12T14:23:15.996 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'31 2020-09-12T14:23:15.996 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'32 2020-09-12T14:23:15.996 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'33 2020-09-12T14:23:15.996 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'34 2020-09-12T14:23:15.996 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'35 2020-09-12T14:23:15.996 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'36 2020-09-12T14:23:15.997 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'37 2020-09-12T14:23:15.997 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'38 2020-09-12T14:23:15.997 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'39 2020-09-12T14:23:15.997 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'40 2020-09-12T14:23:15.997 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'41 2020-09-12T14:23:15.997 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'42 2020-09-12T14:23:15.997 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'43 2020-09-12T14:23:15.997 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'44 2020-09-12T14:23:15.998 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'45 2020-09-12T14:23:15.998 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'46 2020-09-12T14:23:15.998 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'47 2020-09-12T14:23:15.998 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'48 2020-09-12T14:23:15.998 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'49 2020-09-12T14:23:15.998 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'50 2020-09-12T14:23:15.998 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'51 2020-09-12T14:23:15.999 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'52 2020-09-12T14:23:15.999 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'53 2020-09-12T14:23:15.999 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'54 2020-09-12T14:23:15.999 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'55 2020-09-12T14:23:15.999 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'56 2020-09-12T14:23:15.999 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'57 2020-09-12T14:23:15.999 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'58 2020-09-12T14:23:15.999 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'59 2020-09-12T14:23:16.000 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'60 2020-09-12T14:23:16.000 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'61 2020-09-12T14:23:16.000 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'62 2020-09-12T14:23:16.000 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'63 2020-09-12T14:23:16.000 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'64 2020-09-12T14:23:16.000 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'65 2020-09-12T14:23:16.000 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'66 2020-09-12T14:23:16.000 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'67 2020-09-12T14:23:16.001 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'68 2020-09-12T14:23:16.001 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'69 2020-09-12T14:23:16.001 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'70 2020-09-12T14:23:16.001 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'71 2020-09-12T14:23:16.001 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'72 2020-09-12T14:23:16.001 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'73 2020-09-12T14:23:16.001 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'74 2020-09-12T14:23:16.001 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'75 2020-09-12T14:23:16.002 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'76 2020-09-12T14:23:16.002 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'77 2020-09-12T14:23:16.002 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'78 2020-09-12T14:23:16.002 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'79 2020-09-12T14:23:16.002 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'80 2020-09-12T14:23:16.003 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'81 2020-09-12T14:23:16.003 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'82 2020-09-12T14:23:16.003 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'83 2020-09-12T14:23:16.003 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'84 2020-09-12T14:23:16.003 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'85 2020-09-12T14:23:16.003 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'86 2020-09-12T14:23:16.003 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'87 2020-09-12T14:23:16.004 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'88 2020-09-12T14:23:16.004 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'89 2020-09-12T14:23:16.004 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'90 2020-09-12T14:23:16.004 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'91 2020-09-12T14:23:16.004 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'92 2020-09-12T14:23:16.004 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'93 2020-09-12T14:23:16.004 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'94 2020-09-12T14:23:16.005 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'95 2020-09-12T14:23:16.005 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'96 2020-09-12T14:23:16.005 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'97 2020-09-12T14:23:16.005 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'98 2020-09-12T14:23:16.006 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'99 2020-09-12T14:23:16.006 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'100 2020-09-12T14:23:16.006 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'101 2020-09-12T14:23:16.006 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'102 2020-09-12T14:23:16.006 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'103 2020-09-12T14:23:16.006 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'104 2020-09-12T14:23:16.007 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'105 2020-09-12T14:23:16.007 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'106 2020-09-12T14:23:16.007 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'107 2020-09-12T14:23:16.007 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'108 2020-09-12T14:23:16.007 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'109 2020-09-12T14:23:16.007 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'110 2020-09-12T14:23:16.007 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'111 2020-09-12T14:23:16.007 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'112 2020-09-12T14:23:16.008 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'113 2020-09-12T14:23:16.008 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'114 2020-09-12T14:23:16.008 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'115 2020-09-12T14:23:16.008 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'116 2020-09-12T14:23:16.008 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'117 2020-09-12T14:23:16.008 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'118 2020-09-12T14:23:16.008 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'119 2020-09-12T14:23:16.009 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'120 2020-09-12T14:23:16.009 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'121 2020-09-12T14:23:16.009 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'122 2020-09-12T14:23:16.009 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'123 2020-09-12T14:23:16.009 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'124 2020-09-12T14:23:16.009 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'125 2020-09-12T14:23:16.009 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'126 2020-09-12T14:23:16.009 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'127 2020-09-12T14:23:16.010 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'128 2020-09-12T14:23:16.010 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'129 2020-09-12T14:23:16.010 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'130 2020-09-12T14:23:16.010 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'131 2020-09-12T14:23:16.010 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'132 2020-09-12T14:23:16.010 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'133 2020-09-12T14:23:16.010 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'134 2020-09-12T14:23:16.011 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'135 2020-09-12T14:23:16.011 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'136 2020-09-12T14:23:16.011 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'137 2020-09-12T14:23:16.011 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'138 2020-09-12T14:23:16.011 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'139 2020-09-12T14:23:16.011 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'140 2020-09-12T14:23:16.011 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'141 2020-09-12T14:23:16.011 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'142 2020-09-12T14:23:16.012 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'143 2020-09-12T14:23:16.012 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'144 2020-09-12T14:23:16.012 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'145 2020-09-12T14:23:16.012 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'146 2020-09-12T14:23:16.012 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'147 2020-09-12T14:23:16.012 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'148 2020-09-12T14:23:16.012 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to max 2 2020-09-12T14:23:16.013 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to copy log version 34'150 2020-09-12T14:23:16.013 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to copy log version 34'149 2020-09-12T14:23:16.013 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after earliest_dup_version 0 2020-09-12T14:23:16.013 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 33'1 2020-09-12T14:23:16.013 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'2 2020-09-12T14:23:16.013 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'3 2020-09-12T14:23:16.014 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'4 2020-09-12T14:23:16.014 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'5 2020-09-12T14:23:16.014 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'6 2020-09-12T14:23:16.014 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'7 2020-09-12T14:23:16.014 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'8 2020-09-12T14:23:16.014 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'9 2020-09-12T14:23:16.014 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'10 2020-09-12T14:23:16.015 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'11 2020-09-12T14:23:16.015 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'12 2020-09-12T14:23:16.015 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'13 2020-09-12T14:23:16.015 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'14 2020-09-12T14:23:16.015 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'15 2020-09-12T14:23:16.015 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'16 2020-09-12T14:23:16.015 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'17 2020-09-12T14:23:16.016 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'18 2020-09-12T14:23:16.016 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'19 2020-09-12T14:23:16.016 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'20 2020-09-12T14:23:16.016 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'21 2020-09-12T14:23:16.016 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'22 2020-09-12T14:23:16.016 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'23 2020-09-12T14:23:16.016 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'24 2020-09-12T14:23:16.016 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'25 2020-09-12T14:23:16.017 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'26 2020-09-12T14:23:16.017 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'27 2020-09-12T14:23:16.017 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'28 2020-09-12T14:23:16.017 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'29 2020-09-12T14:23:16.017 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'30 2020-09-12T14:23:16.017 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'31 2020-09-12T14:23:16.017 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'32 2020-09-12T14:23:16.017 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'33 2020-09-12T14:23:16.018 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'34 2020-09-12T14:23:16.018 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'35 2020-09-12T14:23:16.018 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'36 2020-09-12T14:23:16.018 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'37 2020-09-12T14:23:16.018 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'38 2020-09-12T14:23:16.018 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'39 2020-09-12T14:23:16.019 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'40 2020-09-12T14:23:16.019 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'41 2020-09-12T14:23:16.019 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'42 2020-09-12T14:23:16.019 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'43 2020-09-12T14:23:16.019 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'44 2020-09-12T14:23:16.019 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'45 2020-09-12T14:23:16.020 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'46 2020-09-12T14:23:16.020 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'47 2020-09-12T14:23:16.020 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'48 2020-09-12T14:23:16.020 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'49 2020-09-12T14:23:16.020 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'50 2020-09-12T14:23:16.020 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'51 2020-09-12T14:23:16.020 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'52 2020-09-12T14:23:16.021 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'53 2020-09-12T14:23:16.021 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'54 2020-09-12T14:23:16.021 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'55 2020-09-12T14:23:16.021 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'56 2020-09-12T14:23:16.021 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'57 2020-09-12T14:23:16.021 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'58 2020-09-12T14:23:16.021 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'59 2020-09-12T14:23:16.021 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'60 2020-09-12T14:23:16.022 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'61 2020-09-12T14:23:16.022 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'62 2020-09-12T14:23:16.022 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'63 2020-09-12T14:23:16.022 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'64 2020-09-12T14:23:16.022 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'65 2020-09-12T14:23:16.022 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'66 2020-09-12T14:23:16.022 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'67 2020-09-12T14:23:16.023 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'68 2020-09-12T14:23:16.023 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'69 2020-09-12T14:23:16.023 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'70 2020-09-12T14:23:16.023 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'71 2020-09-12T14:23:16.023 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'72 2020-09-12T14:23:16.023 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'73 2020-09-12T14:23:16.024 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'74 2020-09-12T14:23:16.024 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'75 2020-09-12T14:23:16.024 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'76 2020-09-12T14:23:16.024 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'77 2020-09-12T14:23:16.024 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'78 2020-09-12T14:23:16.025 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'79 2020-09-12T14:23:16.025 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'80 2020-09-12T14:23:16.025 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'81 2020-09-12T14:23:16.025 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'82 2020-09-12T14:23:16.026 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'83 2020-09-12T14:23:16.026 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'84 2020-09-12T14:23:16.026 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'85 2020-09-12T14:23:16.026 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'86 2020-09-12T14:23:16.026 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'87 2020-09-12T14:23:16.026 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'88 2020-09-12T14:23:16.027 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'89 2020-09-12T14:23:16.027 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'90 2020-09-12T14:23:16.027 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'91 2020-09-12T14:23:16.027 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'92 2020-09-12T14:23:16.027 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'93 2020-09-12T14:23:16.027 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'94 2020-09-12T14:23:16.027 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'95 2020-09-12T14:23:16.028 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'96 2020-09-12T14:23:16.028 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'97 2020-09-12T14:23:16.028 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'98 2020-09-12T14:23:16.028 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'99 2020-09-12T14:23:16.028 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'100 2020-09-12T14:23:16.028 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'101 2020-09-12T14:23:16.028 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'102 2020-09-12T14:23:16.029 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'103 2020-09-12T14:23:16.029 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'104 2020-09-12T14:23:16.029 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'105 2020-09-12T14:23:16.029 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'106 2020-09-12T14:23:16.030 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'107 2020-09-12T14:23:16.030 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'108 2020-09-12T14:23:16.030 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'109 2020-09-12T14:23:16.030 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'110 2020-09-12T14:23:16.030 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'111 2020-09-12T14:23:16.031 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'112 2020-09-12T14:23:16.031 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'113 2020-09-12T14:23:16.031 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'114 2020-09-12T14:23:16.031 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'115 2020-09-12T14:23:16.031 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'116 2020-09-12T14:23:16.031 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'117 2020-09-12T14:23:16.031 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'118 2020-09-12T14:23:16.032 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'119 2020-09-12T14:23:16.032 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'120 2020-09-12T14:23:16.032 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'121 2020-09-12T14:23:16.032 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'122 2020-09-12T14:23:16.032 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'123 2020-09-12T14:23:16.032 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'124 2020-09-12T14:23:16.032 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'125 2020-09-12T14:23:16.032 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'126 2020-09-12T14:23:16.033 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'127 2020-09-12T14:23:16.033 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'128 2020-09-12T14:23:16.040 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'129 2020-09-12T14:23:16.040 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'130 2020-09-12T14:23:16.040 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'131 2020-09-12T14:23:16.040 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'132 2020-09-12T14:23:16.040 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'133 2020-09-12T14:23:16.040 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'134 2020-09-12T14:23:16.041 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'135 2020-09-12T14:23:16.041 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'136 2020-09-12T14:23:16.041 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'137 2020-09-12T14:23:16.041 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'138 2020-09-12T14:23:16.041 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'139 2020-09-12T14:23:16.351 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'140 2020-09-12T14:23:16.352 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'141 2020-09-12T14:23:16.352 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'142 2020-09-12T14:23:16.352 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'143 2020-09-12T14:23:16.352 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'144 2020-09-12T14:23:16.352 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'145 2020-09-12T14:23:16.353 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'146 2020-09-12T14:23:16.353 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'147 2020-09-12T14:23:16.353 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:22:18.396+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'148 2020-09-12T14:23:16.354 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_after v 34'150 2020-09-12T14:23:16.354 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after earliest_dup_version 0 2020-09-12T14:23:16.354 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 33'1 2020-09-12T14:23:16.355 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'2 2020-09-12T14:23:16.356 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'3 2020-09-12T14:23:16.356 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'4 2020-09-12T14:23:16.357 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'5 2020-09-12T14:23:16.357 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'6 2020-09-12T14:23:16.357 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'7 2020-09-12T14:23:16.358 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'8 2020-09-12T14:23:16.358 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'9 2020-09-12T14:23:16.358 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'10 2020-09-12T14:23:16.359 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'11 2020-09-12T14:23:16.359 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'12 2020-09-12T14:23:16.359 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'13 2020-09-12T14:23:16.360 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'14 2020-09-12T14:23:16.361 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'15 2020-09-12T14:23:16.361 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'16 2020-09-12T14:23:16.361 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'17 2020-09-12T14:23:16.361 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'18 2020-09-12T14:23:16.361 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'19 2020-09-12T14:23:16.361 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'20 2020-09-12T14:23:16.361 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'21 2020-09-12T14:23:16.362 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'22 2020-09-12T14:23:16.362 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'23 2020-09-12T14:23:16.362 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'24 2020-09-12T14:23:16.362 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'25 2020-09-12T14:23:16.362 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'26 2020-09-12T14:23:16.362 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'27 2020-09-12T14:23:16.362 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'28 2020-09-12T14:23:16.363 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'29 2020-09-12T14:23:16.363 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'30 2020-09-12T14:23:16.363 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'31 2020-09-12T14:23:16.363 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'32 2020-09-12T14:23:16.363 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'33 2020-09-12T14:23:16.363 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'34 2020-09-12T14:23:16.363 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'35 2020-09-12T14:23:16.364 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'36 2020-09-12T14:23:16.364 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'37 2020-09-12T14:23:16.364 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'38 2020-09-12T14:23:16.364 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'39 2020-09-12T14:23:16.364 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'40 2020-09-12T14:23:16.364 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'41 2020-09-12T14:23:16.364 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'42 2020-09-12T14:23:16.364 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'43 2020-09-12T14:23:16.365 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'44 2020-09-12T14:23:16.365 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'45 2020-09-12T14:23:16.365 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'46 2020-09-12T14:23:16.365 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'47 2020-09-12T14:23:16.365 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'48 2020-09-12T14:23:16.365 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'49 2020-09-12T14:23:16.365 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'50 2020-09-12T14:23:16.366 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'51 2020-09-12T14:23:16.366 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'52 2020-09-12T14:23:16.366 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'53 2020-09-12T14:23:16.366 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'54 2020-09-12T14:23:16.366 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'55 2020-09-12T14:23:16.366 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'56 2020-09-12T14:23:16.367 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'57 2020-09-12T14:23:16.367 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'58 2020-09-12T14:23:16.367 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'59 2020-09-12T14:23:16.367 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'60 2020-09-12T14:23:16.367 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'61 2020-09-12T14:23:16.367 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'62 2020-09-12T14:23:16.368 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'63 2020-09-12T14:23:16.368 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'64 2020-09-12T14:23:16.368 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'65 2020-09-12T14:23:16.368 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'66 2020-09-12T14:23:16.368 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'67 2020-09-12T14:23:16.368 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'68 2020-09-12T14:23:16.368 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'69 2020-09-12T14:23:16.369 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'70 2020-09-12T14:23:16.369 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'71 2020-09-12T14:23:16.369 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'72 2020-09-12T14:23:16.369 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'73 2020-09-12T14:23:16.369 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'74 2020-09-12T14:23:16.369 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'75 2020-09-12T14:23:16.369 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'76 2020-09-12T14:23:16.370 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'77 2020-09-12T14:23:16.370 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'78 2020-09-12T14:23:16.370 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'79 2020-09-12T14:23:16.370 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'80 2020-09-12T14:23:16.370 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'81 2020-09-12T14:23:16.370 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'82 2020-09-12T14:23:16.370 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'83 2020-09-12T14:23:16.371 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'84 2020-09-12T14:23:16.371 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'85 2020-09-12T14:23:16.371 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'86 2020-09-12T14:23:16.371 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'87 2020-09-12T14:23:16.371 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'88 2020-09-12T14:23:16.371 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'89 2020-09-12T14:23:16.371 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'90 2020-09-12T14:23:16.371 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'91 2020-09-12T14:23:16.372 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'92 2020-09-12T14:23:16.372 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'93 2020-09-12T14:23:16.372 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'94 2020-09-12T14:23:16.372 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'95 2020-09-12T14:23:16.372 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'96 2020-09-12T14:23:16.372 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'97 2020-09-12T14:23:16.372 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'98 2020-09-12T14:23:16.373 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'99 2020-09-12T14:23:16.373 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup version 34'100 2020-09-12T14:23:16.373 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'101 2020-09-12T14:23:16.373 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'102 2020-09-12T14:23:16.373 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'103 2020-09-12T14:23:16.373 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'104 2020-09-12T14:23:16.373 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'105 2020-09-12T14:23:16.373 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'106 2020-09-12T14:23:16.374 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'107 2020-09-12T14:23:16.374 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'108 2020-09-12T14:23:16.374 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'109 2020-09-12T14:23:16.374 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'110 2020-09-12T14:23:16.374 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'111 2020-09-12T14:23:16.374 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'112 2020-09-12T14:23:16.374 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'113 2020-09-12T14:23:16.375 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'114 2020-09-12T14:23:16.375 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'115 2020-09-12T14:23:16.375 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'116 2020-09-12T14:23:16.375 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'117 2020-09-12T14:23:16.375 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'118 2020-09-12T14:23:16.375 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'119 2020-09-12T14:23:16.375 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'120 2020-09-12T14:23:16.375 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'121 2020-09-12T14:23:16.376 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'122 2020-09-12T14:23:16.376 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'123 2020-09-12T14:23:16.376 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'124 2020-09-12T14:23:16.376 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'125 2020-09-12T14:23:16.376 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'126 2020-09-12T14:23:16.376 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'127 2020-09-12T14:23:16.376 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'128 2020-09-12T14:23:16.376 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'129 2020-09-12T14:23:16.377 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'130 2020-09-12T14:23:16.377 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'131 2020-09-12T14:23:16.377 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'132 2020-09-12T14:23:16.377 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'133 2020-09-12T14:23:16.377 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'134 2020-09-12T14:23:16.377 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'135 2020-09-12T14:23:16.377 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'136 2020-09-12T14:23:16.377 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'137 2020-09-12T14:23:16.378 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'138 2020-09-12T14:23:16.378 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'139 2020-09-12T14:23:16.378 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'140 2020-09-12T14:23:16.378 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'141 2020-09-12T14:23:16.378 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'142 2020-09-12T14:23:16.378 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'143 2020-09-12T14:23:16.378 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'144 2020-09-12T14:23:16.379 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'145 2020-09-12T14:23:16.379 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'146 2020-09-12T14:23:16.379 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'147 2020-09-12T14:23:16.379 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'148 2020-09-12T14:23:16.379 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'149 2020-09-12T14:23:16.379 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:23:03.836+0000 7f74aabde700 20 copy_up_to/copy_after copy dup from log version 34'150 2020-09-12T14:23:16.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:95: _common_test: rm -f td/osd-backfill-recovery-log/result.log 2020-09-12T14:23:16.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:96: _common_test: '[' 0 '!=' 0 ']' 2020-09-12T14:23:16.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:34: run: teardown td/osd-backfill-recovery-log 2020-09-12T14:23:16.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-backfill-recovery-log 2020-09-12T14:23:16.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:23:16.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-backfill-recovery-log KILL 2020-09-12T14:23:16.381 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:23:16.381 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:23:16.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:23:16.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:23:16.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:23:16.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:23:16.382 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:23:16.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:23:16.382 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:23:16.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:23:16.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:23:16.382 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:23:16.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:23:16.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:23:16.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:23:16.383 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:23:16.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:23:16.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:23:16.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-backfill-recovery-log 2020-09-12T14:23:16.383 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:23:16.384 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:23:16.384 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:23:16.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.98804 2020-09-12T14:23:16.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:23:16.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:23:16.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:31: run: for func in $funcs 2020-09-12T14:23:16.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:32: run: setup td/osd-backfill-recovery-log 2020-09-12T14:23:16.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-backfill-recovery-log 2020-09-12T14:23:16.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-backfill-recovery-log 2020-09-12T14:23:16.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-backfill-recovery-log 2020-09-12T14:23:16.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:23:16.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-backfill-recovery-log KILL 2020-09-12T14:23:16.385 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:23:16.385 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:23:16.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:23:16.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:23:16.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:23:16.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:23:16.386 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:23:16.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:23:16.386 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:23:16.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:23:16.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:23:16.387 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:23:16.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:23:16.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:23:16.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:23:16.387 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:23:16.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:23:16.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:23:16.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-backfill-recovery-log 2020-09-12T14:23:16.388 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:23:16.388 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:23:16.388 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:23:16.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.98804 2020-09-12T14:23:16.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:23:16.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:23:16.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-backfill-recovery-log 2020-09-12T14:23:16.389 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2020-09-12T14:23:16.389 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:23:16.389 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:23:16.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.98804 2020-09-12T14:23:16.389 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2020-09-12T14:23:16.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2020-09-12T14:23:16.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2020-09-12T14:23:16.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-backfill-recovery-log 1' TERM HUP INT 2020-09-12T14:23:16.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:33: run: TEST_recovery_1 td/osd-backfill-recovery-log 2020-09-12T14:23:16.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:121: TEST_recovery_1: local dir=td/osd-backfill-recovery-log 2020-09-12T14:23:16.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:123: TEST_recovery_1: _common_test td/osd-backfill-recovery-log '--osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10' 40 0 40 2020-09-12T14:23:16.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:40: _common_test: local dir=td/osd-backfill-recovery-log 2020-09-12T14:23:16.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:41: _common_test: local 'extra_opts=--osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10' 2020-09-12T14:23:16.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:42: _common_test: local loglen=40 2020-09-12T14:23:16.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:43: _common_test: local dupslen=0 2020-09-12T14:23:16.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:44: _common_test: local objects=40 2020-09-12T14:23:16.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:45: _common_test: local moreobjects=0 2020-09-12T14:23:16.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:47: _common_test: local OSDS=6 2020-09-12T14:23:16.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:49: _common_test: run_mon td/osd-backfill-recovery-log a 2020-09-12T14:23:16.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-backfill-recovery-log 2020-09-12T14:23:16.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2020-09-12T14:23:16.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2020-09-12T14:23:16.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2020-09-12T14:23:16.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-backfill-recovery-log/a 2020-09-12T14:23:16.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-backfill-recovery-log/a --run-dir=td/osd-backfill-recovery-log 2020-09-12T14:23:16.393 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2020-09-12T14:23:16.393 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:23:16.393 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:23:16.394 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:23:16.394 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:23:16.394 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:23:16.394 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:23:16.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-backfill-recovery-log/a '--log-file=td/osd-backfill-recovery-log/$name.log' '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --mon-cluster-log-file=td/osd-backfill-recovery-log/log --run-dir=td/osd-backfill-recovery-log '--pid-file=td/osd-backfill-recovery-log/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 2020-09-12T14:23:16.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: cat 2020-09-12T14:23:16.533 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a fsid 2020-09-12T14:23:16.534 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:23:16.534 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:23:16.534 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=fsid 2020-09-12T14:23:16.535 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .fsid 2020-09-12T14:23:16.535 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:23:16.535 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:23:16.535 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:23:16.535 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:23:16.536 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:23:16.536 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:23:16.536 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.98804/ceph-mon.a.asok 2020-09-12T14:23:16.536 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:23:16.537 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.98804/ceph-mon.a.asok config get fsid 2020-09-12T14:23:16.648 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a mon_host 2020-09-12T14:23:16.649 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:23:16.649 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:23:16.649 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=mon_host 2020-09-12T14:23:16.649 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .mon_host 2020-09-12T14:23:16.649 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:23:16.649 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:23:16.650 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:23:16.650 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:23:16.651 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:23:16.651 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:23:16.651 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.98804/ceph-mon.a.asok 2020-09-12T14:23:16.651 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:23:16.651 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.98804/ceph-mon.a.asok config get mon_host 2020-09-12T14:23:16.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:50: _common_test: run_mgr td/osd-backfill-recovery-log x 2020-09-12T14:23:16.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: local dir=td/osd-backfill-recovery-log 2020-09-12T14:23:16.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: shift 2020-09-12T14:23:16.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: local id=x 2020-09-12T14:23:16.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: shift 2020-09-12T14:23:16.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: local data=td/osd-backfill-recovery-log/x 2020-09-12T14:23:16.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:558: run_mgr: ceph config set mgr mgr/devicehealth/enable_monitoring off --force 2020-09-12T14:23:17.673 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: get_asok_path 2020-09-12T14:23:17.673 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:23:17.673 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:23:17.673 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:23:17.673 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:23:17.674 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:23:17.674 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:23:17.674 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:23:17.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: ceph-mgr --id x --osd_min_pg_log_entries=1 --osd_max_pg_log_entries=2 --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-backfill-recovery-log/x '--log-file=td/osd-backfill-recovery-log/$name.log' '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --run-dir=td/osd-backfill-recovery-log '--pid-file=td/osd-backfill-recovery-log/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:23:17.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:51: _common_test: export CEPH_ARGS 2020-09-12T14:23:17.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:52: _common_test: export 'EXTRA_OPTS= --osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10' 2020-09-12T14:23:17.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:52: _common_test: EXTRA_OPTS=' --osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10' 2020-09-12T14:23:17.698 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: expr 6 - 1 2020-09-12T14:23:17.699 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: seq 0 5 2020-09-12T14:23:17.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:23:17.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 0 2020-09-12T14:23:17.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:23:17.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:23:17.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=0 2020-09-12T14:23:17.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:23:17.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/0 2020-09-12T14:23:17.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:23:17.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:23:17.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:23:17.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:23:17.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/0' 2020-09-12T14:23:17.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/0/journal' 2020-09-12T14:23:17.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:23:17.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10' 2020-09-12T14:23:17.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:23:17.704 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:23:17.704 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:23:17.704 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:23:17.705 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:23:17.705 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:23:17.705 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:23:17.705 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:23:17.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:23:17.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:23:17.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:23:17.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:23:17.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:23:17.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:23:17.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:23:17.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:23:17.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:23:17.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:23:17.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:23:17.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/0 2020-09-12T14:23:17.708 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:23:17.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=c535451a-3c32-4688-8d83-6b6d6d93458c 2020-09-12T14:23:17.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd0 c535451a-3c32-4688-8d83-6b6d6d93458c' 2020-09-12T14:23:17.709 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:23:17.709 INFO:tasks.workunit.client.0.smithi027.stdout:add osd0 c535451a-3c32-4688-8d83-6b6d6d93458c 2020-09-12T14:23:17.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDV2Vxfe/LAKhAAKDl/TjvxUZpFfFpRoT4OBQ== 2020-09-12T14:23:17.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDV2Vxfe/LAKhAAKDl/TjvxUZpFfFpRoT4OBQ=="}' 2020-09-12T14:23:17.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new c535451a-3c32-4688-8d83-6b6d6d93458c -i td/osd-backfill-recovery-log/0/new.json 2020-09-12T14:23:18.113 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:23:18.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/0/new.json 2020-09-12T14:23:18.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 0 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/0 --osd-journal=td/osd-backfill-recovery-log/0/journal --chdir= --osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDV2Vxfe/LAKhAAKDl/TjvxUZpFfFpRoT4OBQ== --osd-uuid c535451a-3c32-4688-8d83-6b6d6d93458c 2020-09-12T14:23:18.151 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:18.151+0000 7fc3a6495ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:18.152 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:18.152+0000 7fc3a6495ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:18.153 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:18.153+0000 7fc3a6495ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:18.154 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:18.153+0000 7fc3a6495ec0 -1 bluestore(td/osd-backfill-recovery-log/0/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/0/block: (2) No such file or directory 2020-09-12T14:23:18.154 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:18.154+0000 7fc3a6495ec0 -1 bluestore(td/osd-backfill-recovery-log/0/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/0/block: (2) No such file or directory 2020-09-12T14:23:18.154 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:18.154+0000 7fc3a6495ec0 -1 bluestore(td/osd-backfill-recovery-log/0/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/0/block: (2) No such file or directory 2020-09-12T14:23:18.190 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:18.189+0000 7fc3a6495ec0 -1 bluestore(td/osd-backfill-recovery-log/0) _read_fsid unparsable uuid 2020-09-12T14:23:22.966 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/0/keyring 2020-09-12T14:23:22.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:23:22.967 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd0 key to auth repository 2020-09-12T14:23:22.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd0 key to auth repository 2020-09-12T14:23:22.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:23:23.517 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.0 2020-09-12T14:23:23.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.0 2020-09-12T14:23:23.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 0 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/0 --osd-journal=td/osd-backfill-recovery-log/0/journal --chdir= --osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:23:23.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:23:23.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:23:23.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:23:23.541 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:23.540+0000 7f7c3dcaaec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:23.543 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:23.542+0000 7f7c3dcaaec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:23.544 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:23.543+0000 7f7c3dcaaec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:23.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 0 2020-09-12T14:23:23.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:23:23.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T14:23:23.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:23:23.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:23:23.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:23:23.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:23:23.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:23:23.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:23:23.918 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:23:24.060 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:24.059+0000 7f7c3dcaaec0 -1 Falling back to public interface 2020-09-12T14:23:24.315 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:23:24.973 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:24.972+0000 7f7c3dcaaec0 -1 osd.0 0 log_to_monitors {default=true} 2020-09-12T14:23:25.316 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:23:25.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:23:25.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:23:25.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:23:25.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:23:25.317 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:23:25.732 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/125374,v1:127.0.0.1:6803/125374] [v2:127.0.0.1:6804/125374,v1:127.0.0.1:6805/125374] exists,up c535451a-3c32-4688-8d83-6b6d6d93458c 2020-09-12T14:23:25.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:23:25.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:23:25.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:23:25.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:23:25.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 1 2020-09-12T14:23:25.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:23:25.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:23:25.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=1 2020-09-12T14:23:25.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:23:25.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/1 2020-09-12T14:23:25.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:23:25.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:23:25.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:23:25.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:23:25.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/1' 2020-09-12T14:23:25.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/1/journal' 2020-09-12T14:23:25.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:23:25.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10' 2020-09-12T14:23:25.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:23:25.735 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:23:25.735 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:23:25.735 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:23:25.736 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:23:25.736 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:23:25.736 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:23:25.736 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:23:25.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:23:25.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:23:25.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:23:25.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:23:25.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:23:25.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:23:25.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:23:25.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:23:25.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:23:25.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:23:25.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:23:25.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/1 2020-09-12T14:23:25.738 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:23:25.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=4cda51ab-33a5-4d67-9e74-e9f11edadc80 2020-09-12T14:23:25.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd1 4cda51ab-33a5-4d67-9e74-e9f11edadc80' 2020-09-12T14:23:25.739 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:23:25.739 INFO:tasks.workunit.client.0.smithi027.stdout:add osd1 4cda51ab-33a5-4d67-9e74-e9f11edadc80 2020-09-12T14:23:25.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDd2VxfkpWTLBAAEm8cQ91wuDMiQVPb0Q1mXA== 2020-09-12T14:23:25.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDd2VxfkpWTLBAAEm8cQ91wuDMiQVPb0Q1mXA=="}' 2020-09-12T14:23:25.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 4cda51ab-33a5-4d67-9e74-e9f11edadc80 -i td/osd-backfill-recovery-log/1/new.json 2020-09-12T14:23:26.404 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:23:26.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/1/new.json 2020-09-12T14:23:26.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 1 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/1 --osd-journal=td/osd-backfill-recovery-log/1/journal --chdir= --osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDd2VxfkpWTLBAAEm8cQ91wuDMiQVPb0Q1mXA== --osd-uuid 4cda51ab-33a5-4d67-9e74-e9f11edadc80 2020-09-12T14:23:26.442 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:26.441+0000 7fa43bc9aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:26.444 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:26.442+0000 7fa43bc9aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:26.445 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:26.444+0000 7fa43bc9aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:26.446 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:26.444+0000 7fa43bc9aec0 -1 bluestore(td/osd-backfill-recovery-log/1/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/1/block: (2) No such file or directory 2020-09-12T14:23:26.446 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:26.444+0000 7fa43bc9aec0 -1 bluestore(td/osd-backfill-recovery-log/1/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/1/block: (2) No such file or directory 2020-09-12T14:23:26.446 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:26.444+0000 7fa43bc9aec0 -1 bluestore(td/osd-backfill-recovery-log/1/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/1/block: (2) No such file or directory 2020-09-12T14:23:26.498 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:26.497+0000 7fa43bc9aec0 -1 bluestore(td/osd-backfill-recovery-log/1) _read_fsid unparsable uuid 2020-09-12T14:23:31.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/1/keyring 2020-09-12T14:23:31.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:23:31.125 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd1 key to auth repository 2020-09-12T14:23:31.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd1 key to auth repository 2020-09-12T14:23:31.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:23:31.673 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.1 2020-09-12T14:23:31.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 1 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/1 --osd-journal=td/osd-backfill-recovery-log/1/journal --chdir= --osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:23:31.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:23:31.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:23:31.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:23:31.675 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T14:23:31.697 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:31.696+0000 7fed2d9f1ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:31.699 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:31.697+0000 7fed2d9f1ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:31.700 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:31.698+0000 7fed2d9f1ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:32.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 1 2020-09-12T14:23:32.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:23:32.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:23:32.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:23:32.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:23:32.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:23:32.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:23:32.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:23:32.083 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:23:32.083 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:23:32.220 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:32.219+0000 7fed2d9f1ec0 -1 Falling back to public interface 2020-09-12T14:23:32.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:23:33.077 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:33.075+0000 7fed2d9f1ec0 -1 osd.1 0 log_to_monitors {default=true} 2020-09-12T14:23:33.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:23:33.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:23:33.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:23:33.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:23:33.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:23:33.495 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:23:33.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:23:34.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:23:34.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:23:34.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 2 2020-09-12T14:23:34.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:23:34.914 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:23:34.914 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:23:35.326 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/126368,v1:127.0.0.1:6811/126368] [v2:127.0.0.1:6812/126368,v1:127.0.0.1:6813/126368] exists,up 4cda51ab-33a5-4d67-9e74-e9f11edadc80 2020-09-12T14:23:35.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:23:35.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:23:35.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:23:35.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:23:35.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 2 2020-09-12T14:23:35.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:23:35.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:23:35.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=2 2020-09-12T14:23:35.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:23:35.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/2 2020-09-12T14:23:35.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:23:35.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:23:35.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:23:35.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:23:35.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/2' 2020-09-12T14:23:35.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/2/journal' 2020-09-12T14:23:35.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:23:35.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10' 2020-09-12T14:23:35.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:23:35.329 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:23:35.330 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:23:35.330 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:23:35.330 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:23:35.330 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:23:35.330 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:23:35.330 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:23:35.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:23:35.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:23:35.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:23:35.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:23:35.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:23:35.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:23:35.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:23:35.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:23:35.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:23:35.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:23:35.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:23:35.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/2 2020-09-12T14:23:35.333 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:23:35.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=23e7d9a7-d8e9-4540-9486-e07134aae3e9 2020-09-12T14:23:35.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd2 23e7d9a7-d8e9-4540-9486-e07134aae3e9' 2020-09-12T14:23:35.333 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:23:35.333 INFO:tasks.workunit.client.0.smithi027.stdout:add osd2 23e7d9a7-d8e9-4540-9486-e07134aae3e9 2020-09-12T14:23:35.342 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDn2VxfNBxlFBAACYzBtsvu+v1O/e3BQmnMMw== 2020-09-12T14:23:35.342 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDn2VxfNBxlFBAACYzBtsvu+v1O/e3BQmnMMw=="}' 2020-09-12T14:23:35.342 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 23e7d9a7-d8e9-4540-9486-e07134aae3e9 -i td/osd-backfill-recovery-log/2/new.json 2020-09-12T14:23:35.893 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:23:35.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/2/new.json 2020-09-12T14:23:35.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 2 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/2 --osd-journal=td/osd-backfill-recovery-log/2/journal --chdir= --osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDn2VxfNBxlFBAACYzBtsvu+v1O/e3BQmnMMw== --osd-uuid 23e7d9a7-d8e9-4540-9486-e07134aae3e9 2020-09-12T14:23:35.934 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:35.930+0000 7fd2f4bbfec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:35.935 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:35.932+0000 7fd2f4bbfec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:35.935 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:35.933+0000 7fd2f4bbfec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:35.935 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:35.933+0000 7fd2f4bbfec0 -1 bluestore(td/osd-backfill-recovery-log/2/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/2/block: (2) No such file or directory 2020-09-12T14:23:35.935 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:35.933+0000 7fd2f4bbfec0 -1 bluestore(td/osd-backfill-recovery-log/2/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/2/block: (2) No such file or directory 2020-09-12T14:23:35.936 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:35.933+0000 7fd2f4bbfec0 -1 bluestore(td/osd-backfill-recovery-log/2/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/2/block: (2) No such file or directory 2020-09-12T14:23:36.050 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:36.049+0000 7fd2f4bbfec0 -1 bluestore(td/osd-backfill-recovery-log/2) _read_fsid unparsable uuid 2020-09-12T14:23:40.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/2/keyring 2020-09-12T14:23:40.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:23:40.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd2 key to auth repository 2020-09-12T14:23:40.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:23:40.855 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd2 key to auth repository 2020-09-12T14:23:41.399 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.2 2020-09-12T14:23:41.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.2 2020-09-12T14:23:41.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:23:41.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 2 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/2 --osd-journal=td/osd-backfill-recovery-log/2/journal --chdir= --osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:23:41.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:23:41.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:23:41.424 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:41.424+0000 7f3e87820ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:41.445 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:41.445+0000 7f3e87820ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:41.447 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:41.446+0000 7f3e87820ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:41.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 2 2020-09-12T14:23:41.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:23:41.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T14:23:41.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:23:41.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:23:41.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:23:41.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:23:41.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:23:41.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:23:41.811 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:23:41.964 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:41.963+0000 7f3e87820ec0 -1 Falling back to public interface 2020-09-12T14:23:42.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:23:42.822 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:42.821+0000 7f3e87820ec0 -1 osd.2 0 log_to_monitors {default=true} 2020-09-12T14:23:43.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:23:43.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:23:43.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:23:43.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:23:43.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:23:43.260 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:23:43.796 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/127388,v1:127.0.0.1:6819/127388] [v2:127.0.0.1:6820/127388,v1:127.0.0.1:6821/127388] exists,up 23e7d9a7-d8e9-4540-9486-e07134aae3e9 2020-09-12T14:23:43.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:23:43.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:23:43.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:23:43.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:23:43.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 3 2020-09-12T14:23:43.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:23:43.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:23:43.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=3 2020-09-12T14:23:43.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:23:43.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/3 2020-09-12T14:23:43.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:23:43.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:23:43.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:23:43.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:23:43.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/3' 2020-09-12T14:23:43.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/3/journal' 2020-09-12T14:23:43.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:23:43.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10' 2020-09-12T14:23:43.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:23:43.799 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:23:43.799 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:23:43.799 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:23:43.799 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:23:43.800 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:23:43.800 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:23:43.800 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:23:43.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:23:43.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:23:43.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:23:43.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:23:43.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:23:43.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:23:43.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:23:43.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:23:43.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:23:43.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:23:43.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:23:43.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/3 2020-09-12T14:23:43.802 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:23:43.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=a723b95d-137e-4c76-a22e-612190d5adb1 2020-09-12T14:23:43.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd3 a723b95d-137e-4c76-a22e-612190d5adb1' 2020-09-12T14:23:43.803 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:23:43.803 INFO:tasks.workunit.client.0.smithi027.stdout:add osd3 a723b95d-137e-4c76-a22e-612190d5adb1 2020-09-12T14:23:43.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDv2VxfayBcMBAAlj8A15rvpJ4ZIr+reRaS3A== 2020-09-12T14:23:43.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDv2VxfayBcMBAAlj8A15rvpJ4ZIr+reRaS3A=="}' 2020-09-12T14:23:43.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new a723b95d-137e-4c76-a22e-612190d5adb1 -i td/osd-backfill-recovery-log/3/new.json 2020-09-12T14:23:44.444 INFO:tasks.workunit.client.0.smithi027.stdout:3 2020-09-12T14:23:44.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/3/new.json 2020-09-12T14:23:44.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 3 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/3 --osd-journal=td/osd-backfill-recovery-log/3/journal --chdir= --osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDv2VxfayBcMBAAlj8A15rvpJ4ZIr+reRaS3A== --osd-uuid a723b95d-137e-4c76-a22e-612190d5adb1 2020-09-12T14:23:44.482 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:44.482+0000 7fee3fa21ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:44.484 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:44.484+0000 7fee3fa21ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:44.486 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:44.485+0000 7fee3fa21ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:44.486 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:44.486+0000 7fee3fa21ec0 -1 bluestore(td/osd-backfill-recovery-log/3/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/3/block: (2) No such file or directory 2020-09-12T14:23:44.487 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:44.486+0000 7fee3fa21ec0 -1 bluestore(td/osd-backfill-recovery-log/3/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/3/block: (2) No such file or directory 2020-09-12T14:23:44.487 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:44.486+0000 7fee3fa21ec0 -1 bluestore(td/osd-backfill-recovery-log/3/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/3/block: (2) No such file or directory 2020-09-12T14:23:44.570 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:44.570+0000 7fee3fa21ec0 -1 bluestore(td/osd-backfill-recovery-log/3) _read_fsid unparsable uuid 2020-09-12T14:23:49.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/3/keyring 2020-09-12T14:23:49.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:23:49.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd3 key to auth repository 2020-09-12T14:23:49.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:23:49.180 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd3 key to auth repository 2020-09-12T14:23:49.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.3 2020-09-12T14:23:49.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 3 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/3 --osd-journal=td/osd-backfill-recovery-log/3/journal --chdir= --osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:23:49.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:23:49.793 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.3 2020-09-12T14:23:49.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:23:49.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:23:49.816 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:49.816+0000 7f3965011ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:49.828 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:49.828+0000 7f3965011ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:49.830 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:49.829+0000 7f3965011ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:50.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 3 2020-09-12T14:23:50.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:23:50.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=3 2020-09-12T14:23:50.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:23:50.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:23:50.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:23:50.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:23:50.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:23:50.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.3 up' 2020-09-12T14:23:50.203 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:23:50.348 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:50.347+0000 7f3965011ec0 -1 Falling back to public interface 2020-09-12T14:23:50.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:23:51.222 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:51.221+0000 7f3965011ec0 -1 osd.3 0 log_to_monitors {default=true} 2020-09-12T14:23:51.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:23:51.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:23:51.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:23:51.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:23:51.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.3 up' 2020-09-12T14:23:51.603 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:23:52.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:23:53.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:23:53.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:23:53.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 2 2020-09-12T14:23:53.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:23:53.023 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:23:53.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.3 up' 2020-09-12T14:23:53.510 INFO:tasks.workunit.client.0.smithi027.stdout:osd.3 up in weight 1 up_from 20 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/128382,v1:127.0.0.1:6827/128382] [v2:127.0.0.1:6828/128382,v1:127.0.0.1:6829/128382] exists,up a723b95d-137e-4c76-a22e-612190d5adb1 2020-09-12T14:23:53.510 INFO:tasks.workunit.client.0.smithi027.stdout:add osd4 9905fb66-11d2-43bc-86d1-f0bf28afebfb 2020-09-12T14:23:53.510 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:23:53.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:23:53.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:23:53.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:23:53.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 4 2020-09-12T14:23:53.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:23:53.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:23:53.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=4 2020-09-12T14:23:53.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:23:53.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/4 2020-09-12T14:23:53.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:23:53.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:23:53.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:23:53.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:23:53.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/4' 2020-09-12T14:23:53.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/4/journal' 2020-09-12T14:23:53.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:23:53.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10' 2020-09-12T14:23:53.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:23:53.516 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:23:53.516 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:23:53.516 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:23:53.517 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:23:53.517 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:23:53.517 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:23:53.517 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:23:53.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:23:53.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:23:53.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:23:53.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:23:53.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:23:53.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:23:53.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:23:53.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:23:53.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:23:53.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:23:53.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:23:53.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/4 2020-09-12T14:23:53.521 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:23:53.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=9905fb66-11d2-43bc-86d1-f0bf28afebfb 2020-09-12T14:23:53.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd4 9905fb66-11d2-43bc-86d1-f0bf28afebfb' 2020-09-12T14:23:53.522 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:23:53.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQD52Vxfc0srHxAAdcr7sOpNzic56LonaSqByA== 2020-09-12T14:23:53.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQD52Vxfc0srHxAAdcr7sOpNzic56LonaSqByA=="}' 2020-09-12T14:23:53.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 9905fb66-11d2-43bc-86d1-f0bf28afebfb -i td/osd-backfill-recovery-log/4/new.json 2020-09-12T14:23:54.113 INFO:tasks.workunit.client.0.smithi027.stdout:4 2020-09-12T14:23:54.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/4/new.json 2020-09-12T14:23:54.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 4 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/4 --osd-journal=td/osd-backfill-recovery-log/4/journal --chdir= --osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQD52Vxfc0srHxAAdcr7sOpNzic56LonaSqByA== --osd-uuid 9905fb66-11d2-43bc-86d1-f0bf28afebfb 2020-09-12T14:23:54.152 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:54.151+0000 7f30d773cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:54.153 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:54.153+0000 7f30d773cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:54.154 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:54.154+0000 7f30d773cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:54.155 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:54.154+0000 7f30d773cec0 -1 bluestore(td/osd-backfill-recovery-log/4/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/4/block: (2) No such file or directory 2020-09-12T14:23:54.155 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:54.154+0000 7f30d773cec0 -1 bluestore(td/osd-backfill-recovery-log/4/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/4/block: (2) No such file or directory 2020-09-12T14:23:54.155 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:54.154+0000 7f30d773cec0 -1 bluestore(td/osd-backfill-recovery-log/4/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/4/block: (2) No such file or directory 2020-09-12T14:23:54.299 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:54.298+0000 7f30d773cec0 -1 bluestore(td/osd-backfill-recovery-log/4) _read_fsid unparsable uuid 2020-09-12T14:23:59.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/4/keyring 2020-09-12T14:23:59.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:23:59.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd4 key to auth repository 2020-09-12T14:23:59.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:23:59.168 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd4 key to auth repository 2020-09-12T14:23:59.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.4 2020-09-12T14:23:59.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 4 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/4 --osd-journal=td/osd-backfill-recovery-log/4/journal --chdir= --osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:23:59.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:23:59.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:23:59.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:23:59.718 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.4 2020-09-12T14:23:59.742 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:59.741+0000 7fd0fed59ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:59.743 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:59.743+0000 7fd0fed59ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:23:59.744 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:23:59.744+0000 7fd0fed59ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:24:00.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 4 2020-09-12T14:24:00.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:24:00.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=4 2020-09-12T14:24:00.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:24:00.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:24:00.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:24:00.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:24:00.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:24:00.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.4 up' 2020-09-12T14:24:00.131 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:24:00.262 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:00.261+0000 7fd0fed59ec0 -1 Falling back to public interface 2020-09-12T14:24:00.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:24:01.108 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:01.106+0000 7fd0fed59ec0 -1 osd.4 0 log_to_monitors {default=true} 2020-09-12T14:24:01.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:24:01.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:24:01.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:24:01.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:24:01.568 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:24:01.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.4 up' 2020-09-12T14:24:02.035 INFO:tasks.workunit.client.0.smithi027.stdout:osd.4 up in weight 1 up_from 25 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/129410,v1:127.0.0.1:6835/129410] [v2:127.0.0.1:6836/129410,v1:127.0.0.1:6837/129410] exists,up 9905fb66-11d2-43bc-86d1-f0bf28afebfb 2020-09-12T14:24:02.035 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:24:02.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:24:02.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:24:02.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:24:02.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 5 2020-09-12T14:24:02.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:24:02.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:24:02.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=5 2020-09-12T14:24:02.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:24:02.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/5 2020-09-12T14:24:02.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:24:02.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:24:02.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:24:02.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:24:02.038 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/5' 2020-09-12T14:24:02.038 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/5/journal' 2020-09-12T14:24:02.038 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:24:02.038 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10' 2020-09-12T14:24:02.038 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:24:02.038 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:24:02.038 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:24:02.039 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:24:02.039 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:24:02.039 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:24:02.039 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:24:02.039 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:24:02.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:24:02.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:24:02.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:24:02.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:24:02.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:24:02.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:24:02.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:24:02.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:24:02.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:24:02.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:24:02.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:24:02.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/5 2020-09-12T14:24:02.042 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:24:02.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=88761839-9fa3-40f9-b838-aab033bcdc1b 2020-09-12T14:24:02.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd5 88761839-9fa3-40f9-b838-aab033bcdc1b' 2020-09-12T14:24:02.043 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:24:02.043 INFO:tasks.workunit.client.0.smithi027.stdout:add osd5 88761839-9fa3-40f9-b838-aab033bcdc1b 2020-09-12T14:24:02.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQAC2lxfmE0FAxAAg69BftoXcNjGof8VFRVFTg== 2020-09-12T14:24:02.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQAC2lxfmE0FAxAAg69BftoXcNjGof8VFRVFTg=="}' 2020-09-12T14:24:02.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 88761839-9fa3-40f9-b838-aab033bcdc1b -i td/osd-backfill-recovery-log/5/new.json 2020-09-12T14:24:02.648 INFO:tasks.workunit.client.0.smithi027.stdout:5 2020-09-12T14:24:02.659 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/5/new.json 2020-09-12T14:24:02.660 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 5 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/5 --osd-journal=td/osd-backfill-recovery-log/5/journal --chdir= --osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQAC2lxfmE0FAxAAg69BftoXcNjGof8VFRVFTg== --osd-uuid 88761839-9fa3-40f9-b838-aab033bcdc1b 2020-09-12T14:24:02.687 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:02.685+0000 7fcba253dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:24:02.742 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:02.740+0000 7fcba253dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:24:02.743 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:02.742+0000 7fcba253dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:24:02.744 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:02.742+0000 7fcba253dec0 -1 bluestore(td/osd-backfill-recovery-log/5/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/5/block: (2) No such file or directory 2020-09-12T14:24:02.744 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:02.742+0000 7fcba253dec0 -1 bluestore(td/osd-backfill-recovery-log/5/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/5/block: (2) No such file or directory 2020-09-12T14:24:02.744 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:02.742+0000 7fcba253dec0 -1 bluestore(td/osd-backfill-recovery-log/5/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/5/block: (2) No such file or directory 2020-09-12T14:24:02.816 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:02.815+0000 7fcba253dec0 -1 bluestore(td/osd-backfill-recovery-log/5) _read_fsid unparsable uuid 2020-09-12T14:24:07.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/5/keyring 2020-09-12T14:24:07.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:24:07.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd5 key to auth repository 2020-09-12T14:24:07.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:24:07.390 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd5 key to auth repository 2020-09-12T14:24:07.945 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.5 2020-09-12T14:24:07.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.5 2020-09-12T14:24:07.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 5 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/5 --osd-journal=td/osd-backfill-recovery-log/5/journal --chdir= --osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:24:07.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:24:07.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:24:07.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:24:07.969 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:07.968+0000 7fea73241ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:24:07.971 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:07.970+0000 7fea73241ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:24:07.972 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:07.970+0000 7fea73241ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:24:08.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 5 2020-09-12T14:24:08.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:24:08.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=5 2020-09-12T14:24:08.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:24:08.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:24:08.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:24:08.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:24:08.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.5 up' 2020-09-12T14:24:08.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:24:08.346 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:24:08.492 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:08.490+0000 7fea73241ec0 -1 Falling back to public interface 2020-09-12T14:24:08.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:24:09.344 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:09.342+0000 7fea73241ec0 -1 osd.5 0 log_to_monitors {default=true} 2020-09-12T14:24:09.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:24:09.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:24:09.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:24:09.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:24:09.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.5 up' 2020-09-12T14:24:09.789 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:24:10.251 INFO:tasks.workunit.client.0.smithi027.stdout:osd.5 up in weight 1 up_from 30 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/130404,v1:127.0.0.1:6843/130404] [v2:127.0.0.1:6844/130404,v1:127.0.0.1:6845/130404] exists,up 88761839-9fa3-40f9-b838-aab033bcdc1b 2020-09-12T14:24:10.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:24:10.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:24:10.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:24:10.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:59: _common_test: create_pool test 1 1 2020-09-12T14:24:10.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test 1 1 2020-09-12T14:24:11.106 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test' already exists 2020-09-12T14:24:11.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:24:12.117 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: seq 1 40 2020-09-12T14:24:12.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:12.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-1 /etc/passwd 2020-09-12T14:24:12.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:12.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-2 /etc/passwd 2020-09-12T14:24:12.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:12.848 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-3 /etc/passwd 2020-09-12T14:24:13.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:13.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-4 /etc/passwd 2020-09-12T14:24:13.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:13.284 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-5 /etc/passwd 2020-09-12T14:24:13.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:13.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-6 /etc/passwd 2020-09-12T14:24:13.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:13.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-7 /etc/passwd 2020-09-12T14:24:13.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:13.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-8 /etc/passwd 2020-09-12T14:24:13.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:13.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-9 /etc/passwd 2020-09-12T14:24:14.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:14.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-10 /etc/passwd 2020-09-12T14:24:14.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:14.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-11 /etc/passwd 2020-09-12T14:24:14.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:14.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-12 /etc/passwd 2020-09-12T14:24:14.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:14.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-13 /etc/passwd 2020-09-12T14:24:14.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:14.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-14 /etc/passwd 2020-09-12T14:24:14.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:14.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-15 /etc/passwd 2020-09-12T14:24:15.009 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:15.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-16 /etc/passwd 2020-09-12T14:24:15.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:15.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-17 /etc/passwd 2020-09-12T14:24:15.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:15.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-18 /etc/passwd 2020-09-12T14:24:15.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:15.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-19 /etc/passwd 2020-09-12T14:24:15.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:15.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-20 /etc/passwd 2020-09-12T14:24:15.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:15.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-21 /etc/passwd 2020-09-12T14:24:16.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:16.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-22 /etc/passwd 2020-09-12T14:24:16.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:16.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-23 /etc/passwd 2020-09-12T14:24:16.291 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:16.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-24 /etc/passwd 2020-09-12T14:24:16.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:16.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-25 /etc/passwd 2020-09-12T14:24:16.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:16.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-26 /etc/passwd 2020-09-12T14:24:16.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:16.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-27 /etc/passwd 2020-09-12T14:24:16.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:16.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-28 /etc/passwd 2020-09-12T14:24:17.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:17.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-29 /etc/passwd 2020-09-12T14:24:17.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:17.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-30 /etc/passwd 2020-09-12T14:24:17.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:17.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-31 /etc/passwd 2020-09-12T14:24:17.498 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:17.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-32 /etc/passwd 2020-09-12T14:24:17.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:17.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-33 /etc/passwd 2020-09-12T14:24:17.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:17.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-34 /etc/passwd 2020-09-12T14:24:17.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:17.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-35 /etc/passwd 2020-09-12T14:24:18.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:18.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-36 /etc/passwd 2020-09-12T14:24:18.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:18.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-37 /etc/passwd 2020-09-12T14:24:18.445 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:18.445 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-38 /etc/passwd 2020-09-12T14:24:18.612 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:18.612 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-39 /etc/passwd 2020-09-12T14:24:18.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:24:18.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-40 /etc/passwd 2020-09-12T14:24:18.934 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:67: _common_test: ceph pg dump pgs --format=json 2020-09-12T14:24:18.934 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:67: _common_test: jq '.pg_stats[0].up[]' 2020-09-12T14:24:19.334 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:24:19.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:67: _common_test: ceph osd out 1 0 2 2020-09-12T14:24:20.017 INFO:tasks.workunit.client.0.smithi027.stderr:osd.1 is already out. osd.0 is already out. osd.2 is already out. 2020-09-12T14:24:20.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:68: _common_test: '[' 0 '!=' 0 ']' 2020-09-12T14:24:20.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:74: _common_test: sleep 1 2020-09-12T14:24:21.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:75: _common_test: wait_for_clean 2020-09-12T14:24:21.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:24:21.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:24:21.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:24:21.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:24:21.029 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 90 .1 2020-09-12T14:24:21.029 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:24:21.030 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:24:21.030 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:24:21.030 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:24:21.030 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:24:21.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:24:21.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:24:21.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:24:21.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:24:21.136 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:24:21.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:24:21.543 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:24:21.544 INFO:tasks.workunit.client.0.smithi027.stderr:2 2020-09-12T14:24:21.544 INFO:tasks.workunit.client.0.smithi027.stderr:3 2020-09-12T14:24:21.544 INFO:tasks.workunit.client.0.smithi027.stderr:4 2020-09-12T14:24:21.544 INFO:tasks.workunit.client.0.smithi027.stderr:5' 2020-09-12T14:24:21.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:24:21.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:24:21.544 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:24:21.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836494 2020-09-12T14:24:21.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836494 2020-09-12T14:24:21.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836494' 2020-09-12T14:24:21.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:24:21.694 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:24:21.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672972 2020-09-12T14:24:21.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672972 2020-09-12T14:24:21.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836494 1-42949672972' 2020-09-12T14:24:21.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:24:21.874 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:24:22.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509450 2020-09-12T14:24:22.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509450 2020-09-12T14:24:22.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836494 1-42949672972 2-64424509450' 2020-09-12T14:24:22.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:24:22.016 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2020-09-12T14:24:22.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=85899345929 2020-09-12T14:24:22.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 85899345929 2020-09-12T14:24:22.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836494 1-42949672972 2-64424509450 3-85899345929' 2020-09-12T14:24:22.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:24:22.159 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2020-09-12T14:24:22.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=107374182406 2020-09-12T14:24:22.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 107374182406 2020-09-12T14:24:22.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836494 1-42949672972 2-64424509450 3-85899345929 4-107374182406' 2020-09-12T14:24:22.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:24:22.300 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2020-09-12T14:24:22.439 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=128849018885 2020-09-12T14:24:22.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 128849018885 2020-09-12T14:24:22.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836494 1-42949672972 2-64424509450 3-85899345929 4-107374182406 5-128849018885' 2020-09-12T14:24:22.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:24:22.441 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836494 2020-09-12T14:24:22.441 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:24:22.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:24:22.443 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836494 2020-09-12T14:24:22.443 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:24:22.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836494 2020-09-12T14:24:22.444 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836494' 2020-09-12T14:24:22.444 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:24:22.444 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836494 2020-09-12T14:24:22.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836492 -lt 21474836494 2020-09-12T14:24:22.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:24:23.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:24:23.844 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:24:24.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836494 -lt 21474836494 2020-09-12T14:24:24.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:24:24.261 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672972 2020-09-12T14:24:24.261 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:24:24.261 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:24:24.262 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672972 2020-09-12T14:24:24.262 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:24:24.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672972 2020-09-12T14:24:24.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672972' 2020-09-12T14:24:24.263 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672972 2020-09-12T14:24:24.263 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:24:24.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672972 -lt 42949672972 2020-09-12T14:24:24.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:24:24.735 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509450 2020-09-12T14:24:24.735 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:24:24.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:24:24.736 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509450 2020-09-12T14:24:24.736 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:24:24.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509450 2020-09-12T14:24:24.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509450' 2020-09-12T14:24:24.737 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509450 2020-09-12T14:24:24.737 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:24:25.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509450 -lt 64424509450 2020-09-12T14:24:25.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:24:25.141 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 3-85899345929 2020-09-12T14:24:25.141 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:24:25.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=3 2020-09-12T14:24:25.142 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 3-85899345929 2020-09-12T14:24:25.142 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:24:25.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=85899345929 2020-09-12T14:24:25.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.3 seq 85899345929' 2020-09-12T14:24:25.143 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.3 seq 85899345929 2020-09-12T14:24:25.144 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 3 2020-09-12T14:24:25.548 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 85899345930 -lt 85899345929 2020-09-12T14:24:25.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:24:25.549 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 4-107374182406 2020-09-12T14:24:25.549 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:24:25.550 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=4 2020-09-12T14:24:25.550 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 4-107374182406 2020-09-12T14:24:25.550 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:24:25.551 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=107374182406 2020-09-12T14:24:25.551 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.4 seq 107374182406' 2020-09-12T14:24:25.551 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.4 seq 107374182406 2020-09-12T14:24:25.552 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 4 2020-09-12T14:24:25.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 107374182407 -lt 107374182406 2020-09-12T14:24:25.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:24:25.990 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 5-128849018885 2020-09-12T14:24:25.990 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:24:25.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=5 2020-09-12T14:24:25.991 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 5-128849018885 2020-09-12T14:24:25.991 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:24:25.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=128849018885 2020-09-12T14:24:25.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.5 seq 128849018885' 2020-09-12T14:24:25.992 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 5 2020-09-12T14:24:25.992 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.5 seq 128849018885 2020-09-12T14:24:26.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 128849018885 -lt 128849018885 2020-09-12T14:24:26.415 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:24:26.415 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:24:26.415 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:24:26.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 1 == 0 2020-09-12T14:24:26.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:24:26.977 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:24:26.977 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:24:26.977 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:24:26.977 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:24:26.978 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:24:26.978 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:24:27.366 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:24:27.367 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:24:27.367 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:24:27.367 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:24:27.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:24:27.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' -1 2020-09-12T14:24:27.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1598: wait_for_clean: loop=0 2020-09-12T14:24:27.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1599: wait_for_clean: num_active_clean=0 2020-09-12T14:24:27.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:24:27.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:24:28.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:24:28.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:24:28.008 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:24:28.008 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:24:28.008 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:24:28.008 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:24:28.009 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:24:28.009 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:24:28.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:24:28.395 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:24:28.396 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:24:28.396 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:24:28.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:24:28.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:24:28.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:24:28.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:24:28.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:24:28.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:24:28.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:24:28.947 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:24:28.947 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:24:29.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=15457 2020-09-12T14:24:29.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 15457 '!=' null 2020-09-12T14:24:29.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:24:29.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:24:29.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:24:29.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:24:29.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:24:29.577 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:24:29.577 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:24:29.577 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:24:29.578 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:24:29.578 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:24:29.578 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:24:29.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:24:29.999 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:24:29.999 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:24:30.000 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:24:30.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:24:30.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:24:30.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:24:30.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:24:30.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:24:30.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:24:30.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:24:30.529 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:24:30.529 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:24:31.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=12874 2020-09-12T14:24:31.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 12874 '!=' null 2020-09-12T14:24:31.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:24:31.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:24:31.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:24:31.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:24:31.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:24:31.174 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:24:31.174 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:24:31.174 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:24:31.175 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:24:31.175 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:24:31.175 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:24:31.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:24:31.572 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:24:31.572 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:24:31.572 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:24:32.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:24:32.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:24:32.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:24:32.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:24:32.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:24:32.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:24:32.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:24:32.103 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:24:32.103 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:24:32.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=12366 2020-09-12T14:24:32.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 12366 '!=' null 2020-09-12T14:24:32.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:24:32.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:24:32.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:24:32.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:24:32.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:24:32.743 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:24:32.743 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:24:32.743 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:24:32.743 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:24:32.743 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:24:32.743 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:24:33.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:24:33.136 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:24:33.137 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:24:33.137 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:24:33.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:24:33.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:24:33.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:24:33.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:24:33.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:24:33.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:24:33.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:24:33.681 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:24:33.681 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:24:34.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=21780 2020-09-12T14:24:34.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 21780 '!=' null 2020-09-12T14:24:34.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:24:34.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:24:34.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:24:34.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:24:34.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:24:34.330 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:24:34.330 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:24:34.330 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:24:34.330 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:24:34.330 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:24:34.331 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:24:34.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:24:34.718 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:24:34.718 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:24:34.719 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:24:35.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:24:35.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:24:35.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:24:35.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:24:35.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:24:35.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:24:35.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:24:35.258 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:24:35.258 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:24:35.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=21255 2020-09-12T14:24:35.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 21255 '!=' null 2020-09-12T14:24:35.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:24:35.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:24:35.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:24:35.892 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:24:35.892 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:24:35.893 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:24:35.893 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:24:35.893 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:24:35.894 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:24:35.894 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:24:35.894 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:24:36.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:24:36.285 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:24:36.286 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:24:36.286 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:24:36.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:24:36.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:24:36.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:24:36.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:24:36.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:24:36.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:24:36.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:24:36.813 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:24:36.814 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:24:37.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=21255 2020-09-12T14:24:37.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 21255 '!=' null 2020-09-12T14:24:37.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:24:37.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:24:37.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:24:37.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:24:37.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:24:37.447 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:24:37.447 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:24:37.447 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:24:37.448 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:24:37.448 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:24:37.448 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:24:37.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:24:37.851 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:24:37.851 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:24:37.851 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:24:38.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 1 2020-09-12T14:24:38.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T14:24:38.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T14:24:38.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:77: _common_test: flush_pg_stats 2020-09-12T14:24:38.378 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:24:38.378 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:24:38.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:24:38.774 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:24:38.774 INFO:tasks.workunit.client.0.smithi027.stderr:2 2020-09-12T14:24:38.775 INFO:tasks.workunit.client.0.smithi027.stderr:3 2020-09-12T14:24:38.775 INFO:tasks.workunit.client.0.smithi027.stderr:4 2020-09-12T14:24:38.775 INFO:tasks.workunit.client.0.smithi027.stderr:5' 2020-09-12T14:24:38.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:24:38.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:24:38.775 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:24:38.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836499 2020-09-12T14:24:38.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836499 2020-09-12T14:24:38.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836499' 2020-09-12T14:24:38.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:24:38.938 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:24:39.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672977 2020-09-12T14:24:39.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672977 2020-09-12T14:24:39.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836499 1-42949672977' 2020-09-12T14:24:39.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:24:39.077 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:24:39.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509455 2020-09-12T14:24:39.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509455 2020-09-12T14:24:39.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836499 1-42949672977 2-64424509455' 2020-09-12T14:24:39.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:24:39.217 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2020-09-12T14:24:39.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=85899345935 2020-09-12T14:24:39.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 85899345935 2020-09-12T14:24:39.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836499 1-42949672977 2-64424509455 3-85899345935' 2020-09-12T14:24:39.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:24:39.357 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2020-09-12T14:24:39.496 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=107374182412 2020-09-12T14:24:39.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 107374182412 2020-09-12T14:24:39.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836499 1-42949672977 2-64424509455 3-85899345935 4-107374182412' 2020-09-12T14:24:39.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:24:39.497 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2020-09-12T14:24:39.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=128849018890 2020-09-12T14:24:39.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 128849018890 2020-09-12T14:24:39.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836499 1-42949672977 2-64424509455 3-85899345935 4-107374182412 5-128849018890' 2020-09-12T14:24:39.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:24:39.638 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836499 2020-09-12T14:24:39.638 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:24:39.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:24:39.639 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836499 2020-09-12T14:24:39.639 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:24:39.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836499 2020-09-12T14:24:39.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836499' 2020-09-12T14:24:39.640 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:24:39.640 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836499 2020-09-12T14:24:40.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836499 -lt 21474836499 2020-09-12T14:24:40.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:24:40.043 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672977 2020-09-12T14:24:40.043 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:24:40.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:24:40.044 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672977 2020-09-12T14:24:40.044 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:24:40.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672977 2020-09-12T14:24:40.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672977' 2020-09-12T14:24:40.045 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672977 2020-09-12T14:24:40.046 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:24:40.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672977 -lt 42949672977 2020-09-12T14:24:40.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:24:40.448 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509455 2020-09-12T14:24:40.449 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:24:40.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:24:40.449 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509455 2020-09-12T14:24:40.450 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:24:40.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509455 2020-09-12T14:24:40.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509455' 2020-09-12T14:24:40.451 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509455 2020-09-12T14:24:40.451 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:24:40.871 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509456 -lt 64424509455 2020-09-12T14:24:40.871 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:24:40.872 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 3-85899345935 2020-09-12T14:24:40.872 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:24:40.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=3 2020-09-12T14:24:40.873 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 3-85899345935 2020-09-12T14:24:40.873 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:24:40.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=85899345935 2020-09-12T14:24:40.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.3 seq 85899345935' 2020-09-12T14:24:40.874 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.3 seq 85899345935 2020-09-12T14:24:40.874 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 3 2020-09-12T14:24:41.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 85899345935 -lt 85899345935 2020-09-12T14:24:41.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:24:41.293 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 4-107374182412 2020-09-12T14:24:41.294 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:24:41.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=4 2020-09-12T14:24:41.294 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 4-107374182412 2020-09-12T14:24:41.295 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:24:41.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=107374182412 2020-09-12T14:24:41.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.4 seq 107374182412' 2020-09-12T14:24:41.296 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.4 seq 107374182412 2020-09-12T14:24:41.296 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 4 2020-09-12T14:24:41.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 107374182412 -lt 107374182412 2020-09-12T14:24:41.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:24:41.716 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 5-128849018890 2020-09-12T14:24:41.717 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:24:41.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=5 2020-09-12T14:24:41.717 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 5-128849018890 2020-09-12T14:24:41.718 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:24:41.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=128849018890 2020-09-12T14:24:41.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.5 seq 128849018890' 2020-09-12T14:24:41.719 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.5 seq 128849018890 2020-09-12T14:24:41.719 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 5 2020-09-12T14:24:42.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 128849018891 -lt 128849018890 2020-09-12T14:24:42.144 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:79: _common_test: ceph pg dump pgs --format=json 2020-09-12T14:24:42.144 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:79: _common_test: jq '.pg_stats[0].up_primary' 2020-09-12T14:24:42.535 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:24:42.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:79: _common_test: newprimary=4 2020-09-12T14:24:42.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:80: _common_test: kill_daemons 2020-09-12T14:24:42.545 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:24:42.545 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:24:42.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:24:42.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:24:42.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:24:42.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:24:42.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:82: _common_test: ERRORS=0 2020-09-12T14:24:42.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:83: _common_test: _objectstore_tool_nodown td/osd-backfill-recovery-log 4 --no-mon-config --pgid 1.0 --op log 2020-09-12T14:24:42.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1195: _objectstore_tool_nodown: local dir=td/osd-backfill-recovery-log 2020-09-12T14:24:42.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: _objectstore_tool_nodown: shift 2020-09-12T14:24:42.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:83: _common_test: tee td/osd-backfill-recovery-log/result.log 2020-09-12T14:24:42.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: _objectstore_tool_nodown: local id=4 2020-09-12T14:24:42.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1198: _objectstore_tool_nodown: shift 2020-09-12T14:24:42.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: _objectstore_tool_nodown: local osd_data=td/osd-backfill-recovery-log/4 2020-09-12T14:24:42.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1202: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-backfill-recovery-log/4 --no-mon-config --pgid 1.0 --op log 2020-09-12T14:24:43.854 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:24:43.855 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_log_t": { 2020-09-12T14:24:43.855 INFO:tasks.workunit.client.0.smithi027.stdout: "head": "34'40", 2020-09-12T14:24:43.855 INFO:tasks.workunit.client.0.smithi027.stdout: "tail": "0'0", 2020-09-12T14:24:43.855 INFO:tasks.workunit.client.0.smithi027.stdout: "log": [ 2020-09-12T14:24:43.855 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.856 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.856 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:c00128d9:::obj-1:head", 2020-09-12T14:24:43.856 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "33'1", 2020-09-12T14:24:43.856 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.856 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4191.0:1", 2020-09-12T14:24:43.856 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.856 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:12.145253+0000", 2020-09-12T14:24:43.857 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.857 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.857 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.857 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.857 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.857 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.857 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.857 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.858 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.858 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.858 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.858 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.858 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.858 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.858 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.859 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.859 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.859 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.859 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:d7942fc0:::obj-2:head", 2020-09-12T14:24:43.859 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'2", 2020-09-12T14:24:43.859 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.859 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4194.0:1", 2020-09-12T14:24:43.859 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.860 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:12.669224+0000", 2020-09-12T14:24:43.860 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.860 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.860 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.860 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.860 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.860 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.861 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.861 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.861 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.861 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.861 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.861 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.861 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.861 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.862 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.862 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.862 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.862 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.862 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:1dda57ab:::obj-3:head", 2020-09-12T14:24:43.862 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'3", 2020-09-12T14:24:43.862 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.863 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4197.0:1", 2020-09-12T14:24:43.863 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.863 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:12.883766+0000", 2020-09-12T14:24:43.863 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.863 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.863 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.863 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.863 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.864 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.864 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.864 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.864 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.864 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.864 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.864 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.864 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.865 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.865 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.865 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.865 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.865 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.865 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:ba1802d0:::obj-4:head", 2020-09-12T14:24:43.865 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'4", 2020-09-12T14:24:43.865 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.866 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4200.0:1", 2020-09-12T14:24:43.866 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.866 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:13.132388+0000", 2020-09-12T14:24:43.866 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.866 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.866 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.866 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.867 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.867 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.867 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.867 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.867 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.867 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.867 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.867 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.868 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.868 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.868 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.868 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.868 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.868 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.868 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:d5d4a582:::obj-5:head", 2020-09-12T14:24:43.868 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'5", 2020-09-12T14:24:43.869 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.869 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4203.0:1", 2020-09-12T14:24:43.869 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.869 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:13.311050+0000", 2020-09-12T14:24:43.869 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.869 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.869 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.869 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.870 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.870 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.870 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.870 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.870 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.870 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.870 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.871 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.871 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.871 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.871 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.871 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.871 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.871 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.871 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:25cac09a:::obj-6:head", 2020-09-12T14:24:43.872 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'6", 2020-09-12T14:24:43.872 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.872 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4206.0:1", 2020-09-12T14:24:43.872 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.872 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:13.436707+0000", 2020-09-12T14:24:43.872 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.872 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.872 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.873 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.873 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.873 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.873 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.873 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.873 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.873 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.874 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.874 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.874 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.874 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.874 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.874 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.874 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.874 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.875 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:b11b7838:::obj-7:head", 2020-09-12T14:24:43.875 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'7", 2020-09-12T14:24:43.875 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.875 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4209.0:1", 2020-09-12T14:24:43.875 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.875 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:13.629612+0000", 2020-09-12T14:24:43.875 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.875 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.876 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.876 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.876 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.876 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.876 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.876 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.876 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.877 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.877 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.877 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.877 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.877 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.877 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.877 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.877 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.878 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.878 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:a0f702de:::obj-8:head", 2020-09-12T14:24:43.878 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'8", 2020-09-12T14:24:43.878 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.878 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4212.0:1", 2020-09-12T14:24:43.878 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.878 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:13.793412+0000", 2020-09-12T14:24:43.878 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.879 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.879 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.879 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.879 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.879 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.879 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.879 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.879 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.880 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.880 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.880 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.880 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.880 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.880 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.880 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.881 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.881 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.881 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:1e5c3677:::obj-9:head", 2020-09-12T14:24:43.881 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'9", 2020-09-12T14:24:43.881 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.881 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4215.0:1", 2020-09-12T14:24:43.881 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.881 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:13.930936+0000", 2020-09-12T14:24:43.882 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.882 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.882 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.882 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.882 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.882 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.882 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.882 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.883 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.883 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.883 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.883 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.883 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.883 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.883 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.883 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.884 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.884 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.884 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:86dc045c:::obj-10:head", 2020-09-12T14:24:43.884 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'10", 2020-09-12T14:24:43.884 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.884 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4218.0:1", 2020-09-12T14:24:43.884 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.885 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:14.048998+0000", 2020-09-12T14:24:43.885 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.885 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.885 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.885 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.885 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.885 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.885 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.886 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.886 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.886 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.886 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.886 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.886 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.886 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.887 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.887 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.887 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.887 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.887 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:1e6eca61:::obj-11:head", 2020-09-12T14:24:43.887 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'11", 2020-09-12T14:24:43.887 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.887 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4221.0:1", 2020-09-12T14:24:43.888 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.888 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:14.216773+0000", 2020-09-12T14:24:43.888 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.888 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.888 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.888 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.888 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.888 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.889 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.889 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.889 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.889 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.889 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.889 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.889 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.889 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.890 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.890 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.890 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.890 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.890 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:18dee445:::obj-12:head", 2020-09-12T14:24:43.890 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'12", 2020-09-12T14:24:43.890 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.891 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4224.0:1", 2020-09-12T14:24:43.891 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.891 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:14.408244+0000", 2020-09-12T14:24:43.891 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.891 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.891 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.891 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.891 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.892 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.892 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.892 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.892 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.892 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.892 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.892 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.892 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.893 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.893 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.893 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.893 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.893 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.893 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:283c5c90:::obj-13:head", 2020-09-12T14:24:43.893 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'13", 2020-09-12T14:24:43.893 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.894 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4227.0:1", 2020-09-12T14:24:43.894 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.894 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:14.543256+0000", 2020-09-12T14:24:43.894 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.894 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.894 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.894 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.895 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.895 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.895 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.895 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.895 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.895 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.895 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.895 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.896 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.896 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.896 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.896 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.896 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.896 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.896 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:cc896bf0:::obj-14:head", 2020-09-12T14:24:43.896 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'14", 2020-09-12T14:24:43.897 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.897 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4230.0:1", 2020-09-12T14:24:43.897 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.897 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:14.734747+0000", 2020-09-12T14:24:43.897 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.897 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.897 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.897 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.898 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.898 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.898 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.898 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.898 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.898 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.898 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.899 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.899 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.899 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.899 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.899 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.899 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.899 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.899 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:a581f807:::obj-15:head", 2020-09-12T14:24:43.900 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'15", 2020-09-12T14:24:43.900 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.900 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4233.0:1", 2020-09-12T14:24:43.900 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.900 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:14.852363+0000", 2020-09-12T14:24:43.900 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.900 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.900 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.901 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.901 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.901 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.901 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.901 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.901 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.901 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.901 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.902 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.902 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.902 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.902 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.902 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.902 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.902 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.903 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:c902ca7c:::obj-16:head", 2020-09-12T14:24:43.903 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'16", 2020-09-12T14:24:43.903 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.903 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4236.0:1", 2020-09-12T14:24:43.903 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.903 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:15.036901+0000", 2020-09-12T14:24:43.903 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.903 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.904 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.904 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.904 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.904 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.904 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.904 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.904 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.904 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.905 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.905 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.905 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.905 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.905 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.905 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.905 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.905 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.906 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:655283c8:::obj-17:head", 2020-09-12T14:24:43.906 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'17", 2020-09-12T14:24:43.906 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.906 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4239.0:1", 2020-09-12T14:24:43.906 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.906 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:15.171019+0000", 2020-09-12T14:24:43.906 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.907 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.907 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.907 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.907 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.907 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.907 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.907 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.907 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.908 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.908 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.908 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.908 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.908 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.908 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.908 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.908 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.909 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.909 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:d759d762:::obj-18:head", 2020-09-12T14:24:43.909 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'18", 2020-09-12T14:24:43.909 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.909 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4242.0:1", 2020-09-12T14:24:43.909 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.909 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:15.363266+0000", 2020-09-12T14:24:43.909 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.910 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.910 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.910 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.910 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.910 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.910 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.910 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.911 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.911 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.911 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.911 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.911 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.911 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.911 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.911 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.912 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.912 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.912 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:1b934a2d:::obj-19:head", 2020-09-12T14:24:43.912 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'19", 2020-09-12T14:24:43.912 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.912 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4245.0:1", 2020-09-12T14:24:43.912 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.912 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:15.580661+0000", 2020-09-12T14:24:43.913 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.913 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.913 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.913 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.913 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.913 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.913 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.913 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.914 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.914 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.914 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.914 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.914 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.914 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.914 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.915 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.915 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.915 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.915 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:1d0ea2ca:::obj-20:head", 2020-09-12T14:24:43.915 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'20", 2020-09-12T14:24:43.915 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.915 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4248.0:1", 2020-09-12T14:24:43.915 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.916 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:15.714591+0000", 2020-09-12T14:24:43.916 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.916 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.916 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.916 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.916 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.916 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.916 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.917 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.917 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.917 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.917 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.917 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.917 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.917 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.918 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.918 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.918 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.918 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.918 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:ab396690:::obj-21:head", 2020-09-12T14:24:43.918 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'21", 2020-09-12T14:24:43.918 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.918 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4251.0:1", 2020-09-12T14:24:43.919 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.919 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:15.907936+0000", 2020-09-12T14:24:43.919 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.919 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.919 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.919 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.919 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.919 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.920 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.920 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.920 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.920 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.920 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.920 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.920 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.920 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.921 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.921 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.921 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.921 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.921 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:254983db:::obj-22:head", 2020-09-12T14:24:43.921 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'22", 2020-09-12T14:24:43.921 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.921 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4254.0:1", 2020-09-12T14:24:43.922 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.922 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:16.042532+0000", 2020-09-12T14:24:43.922 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.922 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.922 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.922 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.922 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.923 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.923 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.923 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.923 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.923 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.923 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.923 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.923 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.924 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.924 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.924 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.924 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.924 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.924 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:e656299f:::obj-23:head", 2020-09-12T14:24:43.924 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'23", 2020-09-12T14:24:43.924 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.925 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4257.0:1", 2020-09-12T14:24:43.925 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.925 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:16.193347+0000", 2020-09-12T14:24:43.925 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.925 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.925 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.925 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.926 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.926 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.926 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.926 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.926 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.926 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.926 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.926 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.927 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.927 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.927 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.927 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.927 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.927 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.927 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:5cdf412b:::obj-24:head", 2020-09-12T14:24:43.927 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'24", 2020-09-12T14:24:43.928 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.928 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4260.0:1", 2020-09-12T14:24:43.928 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.928 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:16.320401+0000", 2020-09-12T14:24:43.928 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.928 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.928 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.928 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.929 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.929 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.929 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.929 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.929 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.929 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.929 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.930 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.930 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.930 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.930 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.930 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.930 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.930 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.930 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:cacae143:::obj-25:head", 2020-09-12T14:24:43.931 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'25", 2020-09-12T14:24:43.931 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.931 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4263.0:1", 2020-09-12T14:24:43.931 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.931 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:16.478102+0000", 2020-09-12T14:24:43.931 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.931 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.931 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.932 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.932 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.932 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.932 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.932 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.932 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.932 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.932 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.933 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.933 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.933 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.933 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.933 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.933 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.933 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.934 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:34b5bff6:::obj-26:head", 2020-09-12T14:24:43.934 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'26", 2020-09-12T14:24:43.934 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.934 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4266.0:1", 2020-09-12T14:24:43.934 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.934 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:16.604805+0000", 2020-09-12T14:24:43.934 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.934 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.935 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.935 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.935 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.935 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.935 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.935 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.935 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.935 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.936 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.936 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.936 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.936 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.936 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.936 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.936 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.936 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.937 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:a354f8f6:::obj-27:head", 2020-09-12T14:24:43.937 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'27", 2020-09-12T14:24:43.937 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.937 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4269.0:1", 2020-09-12T14:24:43.937 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.937 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:16.746337+0000", 2020-09-12T14:24:43.937 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.938 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.938 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.938 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.938 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.938 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.938 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.938 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.938 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.939 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.939 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.939 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.939 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.939 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.939 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.939 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.939 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.940 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.940 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:7421db49:::obj-28:head", 2020-09-12T14:24:43.940 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'28", 2020-09-12T14:24:43.940 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.940 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4272.0:1", 2020-09-12T14:24:43.940 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.940 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:16.871877+0000", 2020-09-12T14:24:43.941 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.941 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.941 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.941 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.941 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.941 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.941 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.941 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.942 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.942 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.942 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.942 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.942 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.942 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.942 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.942 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.943 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.943 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.943 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:54bc79ba:::obj-29:head", 2020-09-12T14:24:43.943 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'29", 2020-09-12T14:24:43.943 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.943 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4275.0:1", 2020-09-12T14:24:43.943 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.943 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:17.047682+0000", 2020-09-12T14:24:43.944 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.944 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.944 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.944 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.944 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.944 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.944 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.945 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.945 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.945 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.945 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.945 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.945 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.945 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.945 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.946 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.946 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.946 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.946 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:01b15595:::obj-30:head", 2020-09-12T14:24:43.946 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'30", 2020-09-12T14:24:43.946 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.946 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4278.0:1", 2020-09-12T14:24:43.946 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.947 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:17.173385+0000", 2020-09-12T14:24:43.947 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.947 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.947 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.947 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.947 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.947 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.948 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.948 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.948 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.948 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.948 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.948 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.948 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.948 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.949 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.949 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.949 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.949 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.949 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:f9a0a8a2:::obj-31:head", 2020-09-12T14:24:43.949 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'31", 2020-09-12T14:24:43.949 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.949 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4281.0:1", 2020-09-12T14:24:43.950 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.950 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:17.332937+0000", 2020-09-12T14:24:43.950 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.950 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.950 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.950 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.950 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.951 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.951 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.951 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.951 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.951 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.951 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.951 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.951 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.952 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.952 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.952 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.952 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.952 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.952 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:1bf2562f:::obj-32:head", 2020-09-12T14:24:43.952 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'32", 2020-09-12T14:24:43.952 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.953 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4284.0:1", 2020-09-12T14:24:43.953 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.953 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:17.525593+0000", 2020-09-12T14:24:43.953 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.953 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.953 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.953 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.953 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.954 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.954 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.954 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.954 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.954 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.954 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.954 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.955 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.955 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.955 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.955 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.955 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.955 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.955 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:b982ae22:::obj-33:head", 2020-09-12T14:24:43.955 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'33", 2020-09-12T14:24:43.956 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.956 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4287.0:1", 2020-09-12T14:24:43.956 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.956 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:17.659823+0000", 2020-09-12T14:24:43.956 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.956 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.956 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.956 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.957 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.957 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.957 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.957 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.957 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.957 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.957 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.957 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.958 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.958 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.958 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.958 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.958 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.958 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.958 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:1ad9f98e:::obj-34:head", 2020-09-12T14:24:43.959 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'34", 2020-09-12T14:24:43.959 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.959 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4290.0:1", 2020-09-12T14:24:43.959 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.959 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:17.845653+0000", 2020-09-12T14:24:43.959 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.959 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.959 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.960 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.960 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.960 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.960 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.960 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.960 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.960 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.960 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.961 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.961 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.961 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.961 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.961 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.961 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.961 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.961 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:941f43ff:::obj-35:head", 2020-09-12T14:24:43.962 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'35", 2020-09-12T14:24:43.962 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.962 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4293.0:1", 2020-09-12T14:24:43.962 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.962 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:17.978199+0000", 2020-09-12T14:24:43.962 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.962 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.962 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.963 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.963 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.963 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.963 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.963 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.963 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.963 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.964 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.964 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.964 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.964 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.964 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.964 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.964 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.964 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.965 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:24ebabe9:::obj-36:head", 2020-09-12T14:24:43.965 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'36", 2020-09-12T14:24:43.965 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:43.965 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4296.0:1", 2020-09-12T14:24:43.965 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:43.965 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:18.171054+0000", 2020-09-12T14:24:43.965 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:43.965 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:43.966 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:43.966 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:43.966 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:43.966 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:43.966 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.966 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.966 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:43.967 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:43.967 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:43.967 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:43.967 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:43.967 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.967 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:43.967 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:43.967 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:43.968 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:43.968 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:5f3a5398:::obj-37:head", 2020-09-12T14:24:43.968 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'37", 2020-09-12T14:24:43.968 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:44.120 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4299.0:1", 2020-09-12T14:24:44.120 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:44.120 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:18.313647+0000", 2020-09-12T14:24:44.121 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:44.121 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:44.121 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:44.121 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:44.121 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:44.121 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:44.121 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:44.122 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:44.122 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:44.122 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:44.122 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:44.122 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:44.122 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:44.122 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:44.123 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:44.123 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:44.123 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:44.123 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:44.123 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:c96cb37e:::obj-38:head", 2020-09-12T14:24:44.123 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'38", 2020-09-12T14:24:44.123 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:44.123 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4302.0:1", 2020-09-12T14:24:44.124 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:44.124 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:18.472224+0000", 2020-09-12T14:24:44.124 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:44.124 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:44.124 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:44.124 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:44.124 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:44.125 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:44.125 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:44.125 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:44.125 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:44.125 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:44.125 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:44.125 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:44.125 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:44.126 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:44.126 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:44.126 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:44.126 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:44.126 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:44.126 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:69e36996:::obj-39:head", 2020-09-12T14:24:44.126 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'39", 2020-09-12T14:24:44.126 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:44.127 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4305.0:1", 2020-09-12T14:24:44.127 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:44.127 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:18.639454+0000", 2020-09-12T14:24:44.127 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:44.127 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:44.127 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:44.127 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:44.128 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:44.128 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:44.128 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:44.128 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:44.128 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:44.128 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:44.128 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:44.128 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:44.129 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:44.129 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:44.129 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:44.129 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:44.129 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:24:44.129 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:24:44.129 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:9f75773e:::obj-40:head", 2020-09-12T14:24:44.129 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'40", 2020-09-12T14:24:44.130 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:24:44.130 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4308.0:1", 2020-09-12T14:24:44.130 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:24:44.130 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:24:18.781533+0000", 2020-09-12T14:24:44.130 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:24:44.130 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:24:44.130 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:24:44.131 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:24:44.131 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:24:44.131 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:24:44.131 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:44.131 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:44.131 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:24:44.131 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:24:44.131 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:24:44.132 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:24:44.132 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:24:44.132 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:44.132 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:44.132 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:44.132 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:24:44.132 INFO:tasks.workunit.client.0.smithi027.stdout: "dups": [] 2020-09-12T14:24:44.132 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:24:44.133 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_missing_t": { 2020-09-12T14:24:44.133 INFO:tasks.workunit.client.0.smithi027.stdout: "missing": [], 2020-09-12T14:24:44.133 INFO:tasks.workunit.client.0.smithi027.stdout: "may_include_deletes": true 2020-09-12T14:24:44.133 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:24:44.133 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:24:44.134 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:84: _common_test: jq '.pg_log_t.log | length' td/osd-backfill-recovery-log/result.log 2020-09-12T14:24:44.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:84: _common_test: LOGLEN=40 2020-09-12T14:24:44.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:85: _common_test: '[' 40 '!=' 40 ']' 2020-09-12T14:24:44.134 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:89: _common_test: jq '.pg_log_t.dups | length' td/osd-backfill-recovery-log/result.log 2020-09-12T14:24:44.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:89: _common_test: DUPSLEN=0 2020-09-12T14:24:44.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:90: _common_test: '[' 0 '!=' 0 ']' 2020-09-12T14:24:44.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:94: _common_test: grep 'copy_up_to\|copy_after' td/osd-backfill-recovery-log/osd.0.log td/osd-backfill-recovery-log/osd.1.log td/osd-backfill-recovery-log/osd.2.log td/osd-backfill-recovery-log/osd.3.log td/osd-backfill-recovery-log/osd.4.log td/osd-backfill-recovery-log/osd.5.log 2020-09-12T14:24:44.135 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after v 0'0 2020-09-12T14:24:44.135 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'40 2020-09-12T14:24:44.135 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'39 2020-09-12T14:24:44.135 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'38 2020-09-12T14:24:44.136 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'37 2020-09-12T14:24:44.136 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'36 2020-09-12T14:24:44.136 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'35 2020-09-12T14:24:44.136 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'34 2020-09-12T14:24:44.136 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'33 2020-09-12T14:24:44.136 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'32 2020-09-12T14:24:44.136 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'31 2020-09-12T14:24:44.137 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'30 2020-09-12T14:24:44.137 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'29 2020-09-12T14:24:44.137 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'28 2020-09-12T14:24:44.137 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'27 2020-09-12T14:24:44.137 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'26 2020-09-12T14:24:44.137 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'25 2020-09-12T14:24:44.137 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'24 2020-09-12T14:24:44.137 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'23 2020-09-12T14:24:44.138 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'22 2020-09-12T14:24:44.138 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'21 2020-09-12T14:24:44.138 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'20 2020-09-12T14:24:44.138 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'19 2020-09-12T14:24:44.138 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'18 2020-09-12T14:24:44.138 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'17 2020-09-12T14:24:44.138 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'16 2020-09-12T14:24:44.138 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'15 2020-09-12T14:24:44.139 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'14 2020-09-12T14:24:44.139 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'13 2020-09-12T14:24:44.139 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'12 2020-09-12T14:24:44.139 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'11 2020-09-12T14:24:44.139 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'10 2020-09-12T14:24:44.139 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'9 2020-09-12T14:24:44.139 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'8 2020-09-12T14:24:44.140 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'7 2020-09-12T14:24:44.140 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'6 2020-09-12T14:24:44.140 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'5 2020-09-12T14:24:44.140 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'4 2020-09-12T14:24:44.141 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'3 2020-09-12T14:24:44.141 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 34'2 2020-09-12T14:24:44.141 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_after copy log version 33'1 2020-09-12T14:24:44.141 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:24:21.649+0000 7f7c1b3ac700 20 copy_up_to/copy_after earliest_dup_version 0 2020-09-12T14:24:44.141 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after v 0'0 2020-09-12T14:24:44.141 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'40 2020-09-12T14:24:44.141 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'39 2020-09-12T14:24:44.142 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'38 2020-09-12T14:24:44.142 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'37 2020-09-12T14:24:44.142 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'36 2020-09-12T14:24:44.142 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'35 2020-09-12T14:24:44.142 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'34 2020-09-12T14:24:44.142 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'33 2020-09-12T14:24:44.142 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'32 2020-09-12T14:24:44.142 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'31 2020-09-12T14:24:44.143 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'30 2020-09-12T14:24:44.143 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'29 2020-09-12T14:24:44.143 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'28 2020-09-12T14:24:44.143 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'27 2020-09-12T14:24:44.143 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'26 2020-09-12T14:24:44.143 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'25 2020-09-12T14:24:44.143 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'24 2020-09-12T14:24:44.144 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'23 2020-09-12T14:24:44.144 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'22 2020-09-12T14:24:44.144 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'21 2020-09-12T14:24:44.144 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'20 2020-09-12T14:24:44.144 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'19 2020-09-12T14:24:44.144 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'18 2020-09-12T14:24:44.144 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'17 2020-09-12T14:24:44.144 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'16 2020-09-12T14:24:44.145 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'15 2020-09-12T14:24:44.145 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'14 2020-09-12T14:24:44.145 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'13 2020-09-12T14:24:44.145 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'12 2020-09-12T14:24:44.145 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'11 2020-09-12T14:24:44.145 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'10 2020-09-12T14:24:44.145 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'9 2020-09-12T14:24:44.145 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'8 2020-09-12T14:24:44.146 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'7 2020-09-12T14:24:44.146 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'6 2020-09-12T14:24:44.146 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'5 2020-09-12T14:24:44.146 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'4 2020-09-12T14:24:44.146 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'3 2020-09-12T14:24:44.146 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'2 2020-09-12T14:24:44.146 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 33'1 2020-09-12T14:24:44.147 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_up_to/copy_after earliest_dup_version 0 2020-09-12T14:24:44.147 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after v 0'0 2020-09-12T14:24:44.147 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'40 2020-09-12T14:24:44.147 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'39 2020-09-12T14:24:44.147 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'38 2020-09-12T14:24:44.147 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'37 2020-09-12T14:24:44.147 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'36 2020-09-12T14:24:44.147 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'35 2020-09-12T14:24:44.148 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'34 2020-09-12T14:24:44.148 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'33 2020-09-12T14:24:44.148 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'32 2020-09-12T14:24:44.148 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'31 2020-09-12T14:24:44.148 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'30 2020-09-12T14:24:44.148 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'29 2020-09-12T14:24:44.148 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'28 2020-09-12T14:24:44.148 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'27 2020-09-12T14:24:44.149 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'26 2020-09-12T14:24:44.149 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'25 2020-09-12T14:24:44.149 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'24 2020-09-12T14:24:44.149 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'23 2020-09-12T14:24:44.149 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'22 2020-09-12T14:24:44.149 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'21 2020-09-12T14:24:44.149 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'20 2020-09-12T14:24:44.149 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'19 2020-09-12T14:24:44.150 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'18 2020-09-12T14:24:44.150 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'17 2020-09-12T14:24:44.150 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'16 2020-09-12T14:24:44.150 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'15 2020-09-12T14:24:44.150 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'14 2020-09-12T14:24:44.150 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'13 2020-09-12T14:24:44.150 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'12 2020-09-12T14:24:44.151 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'11 2020-09-12T14:24:44.151 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'10 2020-09-12T14:24:44.151 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'9 2020-09-12T14:24:44.151 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'8 2020-09-12T14:24:44.151 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'7 2020-09-12T14:24:44.151 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'6 2020-09-12T14:24:44.151 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'5 2020-09-12T14:24:44.151 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'4 2020-09-12T14:24:44.152 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'3 2020-09-12T14:24:44.152 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 34'2 2020-09-12T14:24:44.152 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_after copy log version 33'1 2020-09-12T14:24:44.152 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.4.log:2020-09-12T14:24:21.896+0000 7fd0dc45b700 20 copy_up_to/copy_after earliest_dup_version 0 2020-09-12T14:24:44.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:95: _common_test: rm -f td/osd-backfill-recovery-log/result.log 2020-09-12T14:24:44.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:96: _common_test: '[' 0 '!=' 0 ']' 2020-09-12T14:24:44.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:34: run: teardown td/osd-backfill-recovery-log 2020-09-12T14:24:44.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-backfill-recovery-log 2020-09-12T14:24:44.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:24:44.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-backfill-recovery-log KILL 2020-09-12T14:24:44.154 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:24:44.154 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:24:44.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:24:44.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:24:44.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:24:44.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:24:44.155 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:24:44.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:24:44.155 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:24:44.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:24:44.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:24:44.156 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:24:44.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:24:44.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:24:44.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:24:44.157 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:24:44.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:24:44.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:24:44.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-backfill-recovery-log 2020-09-12T14:24:44.166 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:24:44.166 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:24:44.167 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:24:44.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.98804 2020-09-12T14:24:44.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:24:44.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:24:44.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:31: run: for func in $funcs 2020-09-12T14:24:44.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:32: run: setup td/osd-backfill-recovery-log 2020-09-12T14:24:44.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-backfill-recovery-log 2020-09-12T14:24:44.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-backfill-recovery-log 2020-09-12T14:24:44.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-backfill-recovery-log 2020-09-12T14:24:44.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:24:44.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-backfill-recovery-log KILL 2020-09-12T14:24:44.169 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:24:44.169 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:24:44.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:24:44.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:24:44.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:24:44.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:24:44.170 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:24:44.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:24:44.171 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:24:44.172 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:24:44.172 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:24:44.173 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:24:44.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:24:44.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:24:44.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:24:44.175 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:24:44.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:24:44.177 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:24:44.177 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-backfill-recovery-log 2020-09-12T14:24:44.178 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:24:44.178 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:24:44.178 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:24:44.178 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.98804 2020-09-12T14:24:44.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:24:44.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:24:44.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-backfill-recovery-log 2020-09-12T14:24:44.180 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2020-09-12T14:24:44.180 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:24:44.181 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:24:44.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.98804 2020-09-12T14:24:44.182 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2020-09-12T14:24:44.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2020-09-12T14:24:44.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2020-09-12T14:24:44.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-backfill-recovery-log 1' TERM HUP INT 2020-09-12T14:24:44.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:33: run: TEST_recovery_2 td/osd-backfill-recovery-log 2020-09-12T14:24:44.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:129: TEST_recovery_2: local dir=td/osd-backfill-recovery-log 2020-09-12T14:24:44.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:131: TEST_recovery_2: _common_test td/osd-backfill-recovery-log '--osd_min_pg_log_entries=150 --osd_max_pg_log_entries=150 --osd_pg_log_dups_tracked=3000 --osd_pg_log_trim_min=10' 151 10 141 20 2020-09-12T14:24:44.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:40: _common_test: local dir=td/osd-backfill-recovery-log 2020-09-12T14:24:44.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:41: _common_test: local 'extra_opts=--osd_min_pg_log_entries=150 --osd_max_pg_log_entries=150 --osd_pg_log_dups_tracked=3000 --osd_pg_log_trim_min=10' 2020-09-12T14:24:44.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:42: _common_test: local loglen=151 2020-09-12T14:24:44.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:43: _common_test: local dupslen=10 2020-09-12T14:24:44.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:44: _common_test: local objects=141 2020-09-12T14:24:44.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:45: _common_test: local moreobjects=20 2020-09-12T14:24:44.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:47: _common_test: local OSDS=6 2020-09-12T14:24:44.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:49: _common_test: run_mon td/osd-backfill-recovery-log a 2020-09-12T14:24:44.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-backfill-recovery-log 2020-09-12T14:24:44.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2020-09-12T14:24:44.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2020-09-12T14:24:44.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2020-09-12T14:24:44.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-backfill-recovery-log/a 2020-09-12T14:24:44.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-backfill-recovery-log/a --run-dir=td/osd-backfill-recovery-log 2020-09-12T14:24:44.452 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2020-09-12T14:24:44.452 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:24:44.453 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:24:44.453 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:24:44.453 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:24:44.453 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:24:44.453 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:24:44.454 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-backfill-recovery-log/a '--log-file=td/osd-backfill-recovery-log/$name.log' '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --mon-cluster-log-file=td/osd-backfill-recovery-log/log --run-dir=td/osd-backfill-recovery-log '--pid-file=td/osd-backfill-recovery-log/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 2020-09-12T14:24:44.653 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: cat 2020-09-12T14:24:44.654 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a fsid 2020-09-12T14:24:44.654 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:24:44.654 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:24:44.654 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=fsid 2020-09-12T14:24:44.655 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .fsid 2020-09-12T14:24:44.656 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:24:44.656 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:24:44.656 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:24:44.656 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:24:44.656 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:24:44.656 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:24:44.657 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.98804/ceph-mon.a.asok 2020-09-12T14:24:44.657 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:24:44.657 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.98804/ceph-mon.a.asok config get fsid 2020-09-12T14:24:44.764 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a mon_host 2020-09-12T14:24:44.765 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:24:44.765 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:24:44.765 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=mon_host 2020-09-12T14:24:44.765 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .mon_host 2020-09-12T14:24:44.765 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:24:44.766 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:24:44.766 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:24:44.766 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:24:44.767 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:24:44.767 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:24:44.767 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.98804/ceph-mon.a.asok 2020-09-12T14:24:44.767 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:24:44.767 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.98804/ceph-mon.a.asok config get mon_host 2020-09-12T14:24:44.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:50: _common_test: run_mgr td/osd-backfill-recovery-log x 2020-09-12T14:24:44.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: local dir=td/osd-backfill-recovery-log 2020-09-12T14:24:44.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: shift 2020-09-12T14:24:44.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: local id=x 2020-09-12T14:24:44.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: shift 2020-09-12T14:24:44.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: local data=td/osd-backfill-recovery-log/x 2020-09-12T14:24:44.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:558: run_mgr: ceph config set mgr mgr/devicehealth/enable_monitoring off --force 2020-09-12T14:24:45.776 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: get_asok_path 2020-09-12T14:24:45.776 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:24:45.776 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:24:45.776 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:24:45.776 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:24:45.777 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:24:45.777 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:24:45.777 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:24:45.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: ceph-mgr --id x --osd_min_pg_log_entries=50 --osd_max_pg_log_entries=50 --osd_pg_log_dups_tracked=60 --osd_pg_log_trim_min=10 --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-backfill-recovery-log/x '--log-file=td/osd-backfill-recovery-log/$name.log' '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --run-dir=td/osd-backfill-recovery-log '--pid-file=td/osd-backfill-recovery-log/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:24:45.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:51: _common_test: export CEPH_ARGS 2020-09-12T14:24:45.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:52: _common_test: export 'EXTRA_OPTS= --osd_min_pg_log_entries=150 --osd_max_pg_log_entries=150 --osd_pg_log_dups_tracked=3000 --osd_pg_log_trim_min=10' 2020-09-12T14:24:45.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:52: _common_test: EXTRA_OPTS=' --osd_min_pg_log_entries=150 --osd_max_pg_log_entries=150 --osd_pg_log_dups_tracked=3000 --osd_pg_log_trim_min=10' 2020-09-12T14:24:45.802 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: expr 6 - 1 2020-09-12T14:24:45.803 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: seq 0 5 2020-09-12T14:24:45.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:24:45.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 0 2020-09-12T14:24:45.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:24:45.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:24:45.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=0 2020-09-12T14:24:45.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:24:45.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/0 2020-09-12T14:24:45.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:24:45.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:24:45.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:24:45.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:24:45.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/0' 2020-09-12T14:24:45.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/0/journal' 2020-09-12T14:24:45.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:24:45.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=150 --osd_max_pg_log_entries=150 --osd_pg_log_dups_tracked=3000 --osd_pg_log_trim_min=10' 2020-09-12T14:24:45.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:24:45.807 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:24:45.808 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:24:45.808 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:24:45.808 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:24:45.808 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:24:45.808 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:24:45.808 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:24:45.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:24:45.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:24:45.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:24:45.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:24:45.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:24:45.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:24:45.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:24:45.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:24:45.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:24:45.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:24:45.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:24:45.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/0 2020-09-12T14:24:45.811 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:24:45.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=8baa7fd5-36bf-4aee-9509-2e0edd835853 2020-09-12T14:24:45.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd0 8baa7fd5-36bf-4aee-9509-2e0edd835853' 2020-09-12T14:24:45.812 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:24:45.812 INFO:tasks.workunit.client.0.smithi027.stdout:add osd0 8baa7fd5-36bf-4aee-9509-2e0edd835853 2020-09-12T14:24:45.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQAt2lxfr4H+MBAAtIuGhMhOUqDJdydsXUDLeA== 2020-09-12T14:24:45.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQAt2lxfr4H+MBAAtIuGhMhOUqDJdydsXUDLeA=="}' 2020-09-12T14:24:45.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 8baa7fd5-36bf-4aee-9509-2e0edd835853 -i td/osd-backfill-recovery-log/0/new.json 2020-09-12T14:24:46.196 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:24:46.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/0/new.json 2020-09-12T14:24:46.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 0 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/0 --osd-journal=td/osd-backfill-recovery-log/0/journal --chdir= --osd_min_pg_log_entries=150 --osd_max_pg_log_entries=150 --osd_pg_log_dups_tracked=3000 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQAt2lxfr4H+MBAAtIuGhMhOUqDJdydsXUDLeA== --osd-uuid 8baa7fd5-36bf-4aee-9509-2e0edd835853 2020-09-12T14:24:46.234 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:46.233+0000 7fed2da79ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:24:46.236 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:46.235+0000 7fed2da79ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:24:46.237 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:46.236+0000 7fed2da79ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:24:46.238 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:46.236+0000 7fed2da79ec0 -1 bluestore(td/osd-backfill-recovery-log/0/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/0/block: (2) No such file or directory 2020-09-12T14:24:46.238 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:46.236+0000 7fed2da79ec0 -1 bluestore(td/osd-backfill-recovery-log/0/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/0/block: (2) No such file or directory 2020-09-12T14:24:46.238 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:46.236+0000 7fed2da79ec0 -1 bluestore(td/osd-backfill-recovery-log/0/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/0/block: (2) No such file or directory 2020-09-12T14:24:46.302 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:46.301+0000 7fed2da79ec0 -1 bluestore(td/osd-backfill-recovery-log/0) _read_fsid unparsable uuid 2020-09-12T14:24:50.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/0/keyring 2020-09-12T14:24:50.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:24:50.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd0 key to auth repository 2020-09-12T14:24:50.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:24:50.922 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd0 key to auth repository 2020-09-12T14:24:51.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.0 2020-09-12T14:24:51.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 0 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/0 --osd-journal=td/osd-backfill-recovery-log/0/journal --chdir= --osd_min_pg_log_entries=150 --osd_max_pg_log_entries=150 --osd_pg_log_dups_tracked=3000 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:24:51.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:24:51.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:24:51.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:24:51.476 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.0 2020-09-12T14:24:51.499 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:51.498+0000 7f884dd3dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:24:51.501 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:51.500+0000 7f884dd3dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:24:51.502 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:51.501+0000 7f884dd3dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:24:51.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 0 2020-09-12T14:24:51.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:24:51.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T14:24:51.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:24:51.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:24:51.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:24:51.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:24:51.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:24:51.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:24:51.913 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:24:52.023 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:52.022+0000 7f884dd3dec0 -1 Falling back to public interface 2020-09-12T14:24:52.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:24:52.862 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:52.861+0000 7f884dd3dec0 -1 osd.0 0 log_to_monitors {default=true} 2020-09-12T14:24:53.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:24:53.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:24:53.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:24:53.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:24:53.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:24:53.320 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:24:53.737 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/134324,v1:127.0.0.1:6803/134324] [v2:127.0.0.1:6804/134324,v1:127.0.0.1:6805/134324] exists,up 8baa7fd5-36bf-4aee-9509-2e0edd835853 2020-09-12T14:24:53.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:24:53.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:24:53.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:24:53.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:24:53.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 1 2020-09-12T14:24:53.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:24:53.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:24:53.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=1 2020-09-12T14:24:53.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:24:53.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/1 2020-09-12T14:24:53.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:24:53.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:24:53.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:24:53.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:24:53.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/1' 2020-09-12T14:24:53.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/1/journal' 2020-09-12T14:24:53.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:24:53.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=150 --osd_max_pg_log_entries=150 --osd_pg_log_dups_tracked=3000 --osd_pg_log_trim_min=10' 2020-09-12T14:24:53.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:24:53.740 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:24:53.740 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:24:53.741 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:24:53.741 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:24:53.741 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:24:53.741 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:24:53.741 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:24:53.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:24:53.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:24:53.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:24:53.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:24:53.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:24:53.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:24:53.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:24:53.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:24:53.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:24:53.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:24:53.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:24:53.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/1 2020-09-12T14:24:53.743 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:24:53.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=453588e4-d04f-4faf-8eef-b00b99355b0d 2020-09-12T14:24:53.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd1 453588e4-d04f-4faf-8eef-b00b99355b0d' 2020-09-12T14:24:53.744 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:24:53.744 INFO:tasks.workunit.client.0.smithi027.stdout:add osd1 453588e4-d04f-4faf-8eef-b00b99355b0d 2020-09-12T14:24:53.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQA12lxfSp3iLBAA13dB12li+yf2H4Osv1L27w== 2020-09-12T14:24:53.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQA12lxfSp3iLBAA13dB12li+yf2H4Osv1L27w=="}' 2020-09-12T14:24:53.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 453588e4-d04f-4faf-8eef-b00b99355b0d -i td/osd-backfill-recovery-log/1/new.json 2020-09-12T14:24:54.252 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:24:54.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/1/new.json 2020-09-12T14:24:54.264 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 1 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/1 --osd-journal=td/osd-backfill-recovery-log/1/journal --chdir= --osd_min_pg_log_entries=150 --osd_max_pg_log_entries=150 --osd_pg_log_dups_tracked=3000 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQA12lxfSp3iLBAA13dB12li+yf2H4Osv1L27w== --osd-uuid 453588e4-d04f-4faf-8eef-b00b99355b0d 2020-09-12T14:24:54.291 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:54.289+0000 7f11ef6bbec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:24:54.292 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:54.291+0000 7f11ef6bbec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:24:54.293 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:54.292+0000 7f11ef6bbec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:24:54.294 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:54.293+0000 7f11ef6bbec0 -1 bluestore(td/osd-backfill-recovery-log/1/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/1/block: (2) No such file or directory 2020-09-12T14:24:54.294 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:54.293+0000 7f11ef6bbec0 -1 bluestore(td/osd-backfill-recovery-log/1/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/1/block: (2) No such file or directory 2020-09-12T14:24:54.294 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:54.293+0000 7f11ef6bbec0 -1 bluestore(td/osd-backfill-recovery-log/1/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/1/block: (2) No such file or directory 2020-09-12T14:24:54.360 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:54.359+0000 7f11ef6bbec0 -1 bluestore(td/osd-backfill-recovery-log/1) _read_fsid unparsable uuid 2020-09-12T14:24:59.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/1/keyring 2020-09-12T14:24:59.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:24:59.187 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd1 key to auth repository 2020-09-12T14:24:59.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd1 key to auth repository 2020-09-12T14:24:59.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:24:59.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.1 2020-09-12T14:24:59.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:24:59.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 1 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/1 --osd-journal=td/osd-backfill-recovery-log/1/journal --chdir= --osd_min_pg_log_entries=150 --osd_max_pg_log_entries=150 --osd_pg_log_dups_tracked=3000 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:24:59.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:24:59.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:24:59.736 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T14:24:59.758 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:59.758+0000 7f0843731ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:24:59.760 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:59.760+0000 7f0843731ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:24:59.761 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:24:59.761+0000 7f0843731ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:00.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 1 2020-09-12T14:25:00.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:25:00.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:25:00.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:25:00.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:25:00.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:25:00.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:25:00.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:25:00.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:25:00.137 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:25:00.279 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:00.278+0000 7f0843731ec0 -1 Falling back to public interface 2020-09-12T14:25:00.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:25:01.136 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:01.135+0000 7f0843731ec0 -1 osd.1 0 log_to_monitors {default=true} 2020-09-12T14:25:01.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:25:01.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:25:01.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:25:01.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:25:01.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:25:01.572 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:25:01.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:25:02.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:25:02.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:25:02.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 2 2020-09-12T14:25:02.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:25:02.990 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:25:02.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:25:03.401 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/135326,v1:127.0.0.1:6811/135326] [v2:127.0.0.1:6812/135326,v1:127.0.0.1:6813/135326] exists,up 453588e4-d04f-4faf-8eef-b00b99355b0d 2020-09-12T14:25:03.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:25:03.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:25:03.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:25:03.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:25:03.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 2 2020-09-12T14:25:03.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:25:03.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:25:03.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=2 2020-09-12T14:25:03.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:25:03.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/2 2020-09-12T14:25:03.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:25:03.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:25:03.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:25:03.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:25:03.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/2' 2020-09-12T14:25:03.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/2/journal' 2020-09-12T14:25:03.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:25:03.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=150 --osd_max_pg_log_entries=150 --osd_pg_log_dups_tracked=3000 --osd_pg_log_trim_min=10' 2020-09-12T14:25:03.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:25:03.405 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:25:03.405 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:25:03.405 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:25:03.405 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:25:03.405 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:25:03.405 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:25:03.405 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:25:03.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:25:03.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:25:03.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:25:03.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:25:03.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:25:03.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:25:03.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:25:03.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:25:03.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:25:03.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:25:03.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:25:03.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/2 2020-09-12T14:25:03.408 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:25:03.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=8281bee7-1151-4522-a42a-d10f91d9945c 2020-09-12T14:25:03.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd2 8281bee7-1151-4522-a42a-d10f91d9945c' 2020-09-12T14:25:03.408 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:25:03.408 INFO:tasks.workunit.client.0.smithi027.stdout:add osd2 8281bee7-1151-4522-a42a-d10f91d9945c 2020-09-12T14:25:03.417 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQA/2lxfsZ3XGBAAewRTkwpW8wFjNDRLX5CPyA== 2020-09-12T14:25:03.417 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQA/2lxfsZ3XGBAAewRTkwpW8wFjNDRLX5CPyA=="}' 2020-09-12T14:25:03.417 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 8281bee7-1151-4522-a42a-d10f91d9945c -i td/osd-backfill-recovery-log/2/new.json 2020-09-12T14:25:03.916 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:25:03.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/2/new.json 2020-09-12T14:25:03.928 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 2 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/2 --osd-journal=td/osd-backfill-recovery-log/2/journal --chdir= --osd_min_pg_log_entries=150 --osd_max_pg_log_entries=150 --osd_pg_log_dups_tracked=3000 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQA/2lxfsZ3XGBAAewRTkwpW8wFjNDRLX5CPyA== --osd-uuid 8281bee7-1151-4522-a42a-d10f91d9945c 2020-09-12T14:25:03.954 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:03.954+0000 7f520d81fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:03.956 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:03.956+0000 7f520d81fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:03.958 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:03.957+0000 7f520d81fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:03.958 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:03.958+0000 7f520d81fec0 -1 bluestore(td/osd-backfill-recovery-log/2/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/2/block: (2) No such file or directory 2020-09-12T14:25:03.958 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:03.958+0000 7f520d81fec0 -1 bluestore(td/osd-backfill-recovery-log/2/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/2/block: (2) No such file or directory 2020-09-12T14:25:03.958 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:03.958+0000 7f520d81fec0 -1 bluestore(td/osd-backfill-recovery-log/2/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/2/block: (2) No such file or directory 2020-09-12T14:25:04.018 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:04.018+0000 7f520d81fec0 -1 bluestore(td/osd-backfill-recovery-log/2) _read_fsid unparsable uuid 2020-09-12T14:25:09.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/2/keyring 2020-09-12T14:25:09.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:25:09.127 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd2 key to auth repository 2020-09-12T14:25:09.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd2 key to auth repository 2020-09-12T14:25:09.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:25:09.679 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.2 2020-09-12T14:25:09.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.2 2020-09-12T14:25:09.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:25:09.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 2 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/2 --osd-journal=td/osd-backfill-recovery-log/2/journal --chdir= --osd_min_pg_log_entries=150 --osd_max_pg_log_entries=150 --osd_pg_log_dups_tracked=3000 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:25:09.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:25:09.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:25:09.704 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:09.703+0000 7fec765e3ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:09.725 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:09.724+0000 7fec765e3ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:09.725 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:09.725+0000 7fec765e3ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:10.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 2 2020-09-12T14:25:10.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:25:10.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T14:25:10.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:25:10.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:25:10.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:25:10.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:25:10.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:25:10.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:25:10.140 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:25:10.242 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:10.241+0000 7fec765e3ec0 -1 Falling back to public interface 2020-09-12T14:25:10.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:25:11.093 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:11.092+0000 7fec765e3ec0 -1 osd.2 0 log_to_monitors {default=true} 2020-09-12T14:25:11.547 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:25:11.547 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:25:11.547 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:25:11.548 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:25:11.548 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:25:11.548 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:25:12.052 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/136346,v1:127.0.0.1:6819/136346] [v2:127.0.0.1:6820/136346,v1:127.0.0.1:6821/136346] exists,up 8281bee7-1151-4522-a42a-d10f91d9945c 2020-09-12T14:25:12.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:25:12.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:25:12.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:25:12.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:25:12.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 3 2020-09-12T14:25:12.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:25:12.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:25:12.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=3 2020-09-12T14:25:12.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:25:12.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/3 2020-09-12T14:25:12.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:25:12.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:25:12.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:25:12.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:25:12.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/3' 2020-09-12T14:25:12.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/3/journal' 2020-09-12T14:25:12.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:25:12.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=150 --osd_max_pg_log_entries=150 --osd_pg_log_dups_tracked=3000 --osd_pg_log_trim_min=10' 2020-09-12T14:25:12.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:25:12.056 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:25:12.056 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:25:12.056 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:25:12.056 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:25:12.057 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:25:12.057 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:25:12.057 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:25:12.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:25:12.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:25:12.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:25:12.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:25:12.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:25:12.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:25:12.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:25:12.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:25:12.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:25:12.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:25:12.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:25:12.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/3 2020-09-12T14:25:12.059 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:25:12.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=c031b6b4-87f9-4f90-b335-07211e63c2da 2020-09-12T14:25:12.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd3 c031b6b4-87f9-4f90-b335-07211e63c2da' 2020-09-12T14:25:12.060 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:25:12.060 INFO:tasks.workunit.client.0.smithi027.stdout:add osd3 c031b6b4-87f9-4f90-b335-07211e63c2da 2020-09-12T14:25:12.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQBI2lxfJC0dBBAAhukGDeWPZkmokiYeX+rVGg== 2020-09-12T14:25:12.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQBI2lxfJC0dBBAAhukGDeWPZkmokiYeX+rVGg=="}' 2020-09-12T14:25:12.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new c031b6b4-87f9-4f90-b335-07211e63c2da -i td/osd-backfill-recovery-log/3/new.json 2020-09-12T14:25:12.687 INFO:tasks.workunit.client.0.smithi027.stdout:3 2020-09-12T14:25:12.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/3/new.json 2020-09-12T14:25:12.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 3 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/3 --osd-journal=td/osd-backfill-recovery-log/3/journal --chdir= --osd_min_pg_log_entries=150 --osd_max_pg_log_entries=150 --osd_pg_log_dups_tracked=3000 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQBI2lxfJC0dBBAAhukGDeWPZkmokiYeX+rVGg== --osd-uuid c031b6b4-87f9-4f90-b335-07211e63c2da 2020-09-12T14:25:12.726 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:12.725+0000 7f51215b1ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:12.727 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:12.727+0000 7f51215b1ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:12.728 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:12.728+0000 7f51215b1ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:12.729 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:12.729+0000 7f51215b1ec0 -1 bluestore(td/osd-backfill-recovery-log/3/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/3/block: (2) No such file or directory 2020-09-12T14:25:12.729 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:12.729+0000 7f51215b1ec0 -1 bluestore(td/osd-backfill-recovery-log/3/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/3/block: (2) No such file or directory 2020-09-12T14:25:12.730 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:12.729+0000 7f51215b1ec0 -1 bluestore(td/osd-backfill-recovery-log/3/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/3/block: (2) No such file or directory 2020-09-12T14:25:12.767 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:12.767+0000 7f51215b1ec0 -1 bluestore(td/osd-backfill-recovery-log/3) _read_fsid unparsable uuid 2020-09-12T14:25:17.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/3/keyring 2020-09-12T14:25:17.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:25:17.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd3 key to auth repository 2020-09-12T14:25:17.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:25:17.981 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd3 key to auth repository 2020-09-12T14:25:18.526 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.3 2020-09-12T14:25:18.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.3 2020-09-12T14:25:18.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:25:18.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 3 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/3 --osd-journal=td/osd-backfill-recovery-log/3/journal --chdir= --osd_min_pg_log_entries=150 --osd_max_pg_log_entries=150 --osd_pg_log_dups_tracked=3000 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:25:18.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:25:18.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:25:18.551 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:18.549+0000 7fbb48a7cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:18.630 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:18.628+0000 7fbb48a7cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:18.630 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:18.629+0000 7fbb48a7cec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:18.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 3 2020-09-12T14:25:18.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:25:18.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=3 2020-09-12T14:25:18.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:25:18.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:25:18.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:25:18.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:25:18.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:25:18.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.3 up' 2020-09-12T14:25:18.940 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:25:19.146 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:19.144+0000 7fbb48a7cec0 -1 Falling back to public interface 2020-09-12T14:25:19.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:25:20.121 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:20.120+0000 7fbb48a7cec0 -1 osd.3 0 log_to_monitors {default=true} 2020-09-12T14:25:20.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:25:20.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:25:20.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:25:20.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:25:20.339 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:25:20.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.3 up' 2020-09-12T14:25:20.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:25:21.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:25:21.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:25:21.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 2 2020-09-12T14:25:21.803 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:25:21.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:25:21.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.3 up' 2020-09-12T14:25:22.220 INFO:tasks.workunit.client.0.smithi027.stdout:osd.3 up in weight 1 up_from 20 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/137358,v1:127.0.0.1:6827/137358] [v2:127.0.0.1:6828/137358,v1:127.0.0.1:6829/137358] exists,up c031b6b4-87f9-4f90-b335-07211e63c2da 2020-09-12T14:25:22.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:25:22.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:25:22.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:25:22.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:25:22.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 4 2020-09-12T14:25:22.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:25:22.222 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:25:22.222 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=4 2020-09-12T14:25:22.222 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:25:22.222 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/4 2020-09-12T14:25:22.222 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:25:22.222 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:25:22.222 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:25:22.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:25:22.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/4' 2020-09-12T14:25:22.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/4/journal' 2020-09-12T14:25:22.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:25:22.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=150 --osd_max_pg_log_entries=150 --osd_pg_log_dups_tracked=3000 --osd_pg_log_trim_min=10' 2020-09-12T14:25:22.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:25:22.223 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:25:22.223 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:25:22.224 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:25:22.224 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:25:22.224 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:25:22.224 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:25:22.224 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:25:22.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:25:22.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:25:22.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:25:22.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:25:22.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:25:22.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:25:22.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:25:22.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:25:22.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:25:22.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:25:22.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:25:22.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/4 2020-09-12T14:25:22.226 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:25:22.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=02cecd8c-7673-4267-859f-88af9093e38a 2020-09-12T14:25:22.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd4 02cecd8c-7673-4267-859f-88af9093e38a' 2020-09-12T14:25:22.227 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:25:22.227 INFO:tasks.workunit.client.0.smithi027.stdout:add osd4 02cecd8c-7673-4267-859f-88af9093e38a 2020-09-12T14:25:22.236 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQBS2lxfIBUSDhAARDdi9KZ7UlFbmvbh4xnuOw== 2020-09-12T14:25:22.236 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQBS2lxfIBUSDhAARDdi9KZ7UlFbmvbh4xnuOw=="}' 2020-09-12T14:25:22.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 02cecd8c-7673-4267-859f-88af9093e38a -i td/osd-backfill-recovery-log/4/new.json 2020-09-12T14:25:22.760 INFO:tasks.workunit.client.0.smithi027.stdout:4 2020-09-12T14:25:22.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/4/new.json 2020-09-12T14:25:22.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 4 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/4 --osd-journal=td/osd-backfill-recovery-log/4/journal --chdir= --osd_min_pg_log_entries=150 --osd_max_pg_log_entries=150 --osd_pg_log_dups_tracked=3000 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQBS2lxfIBUSDhAARDdi9KZ7UlFbmvbh4xnuOw== --osd-uuid 02cecd8c-7673-4267-859f-88af9093e38a 2020-09-12T14:25:22.799 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:22.797+0000 7fb27e42aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:22.800 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:22.799+0000 7fb27e42aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:22.801 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:22.800+0000 7fb27e42aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:22.802 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:22.801+0000 7fb27e42aec0 -1 bluestore(td/osd-backfill-recovery-log/4/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/4/block: (2) No such file or directory 2020-09-12T14:25:22.802 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:22.801+0000 7fb27e42aec0 -1 bluestore(td/osd-backfill-recovery-log/4/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/4/block: (2) No such file or directory 2020-09-12T14:25:22.802 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:22.801+0000 7fb27e42aec0 -1 bluestore(td/osd-backfill-recovery-log/4/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/4/block: (2) No such file or directory 2020-09-12T14:25:22.896 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:22.894+0000 7fb27e42aec0 -1 bluestore(td/osd-backfill-recovery-log/4) _read_fsid unparsable uuid 2020-09-12T14:25:27.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/4/keyring 2020-09-12T14:25:27.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:25:27.159 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd4 key to auth repository 2020-09-12T14:25:27.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd4 key to auth repository 2020-09-12T14:25:27.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:25:27.713 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.4 2020-09-12T14:25:27.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.4 2020-09-12T14:25:27.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 4 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/4 --osd-journal=td/osd-backfill-recovery-log/4/journal --chdir= --osd_min_pg_log_entries=150 --osd_max_pg_log_entries=150 --osd_pg_log_dups_tracked=3000 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:25:27.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:25:27.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:25:27.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:25:27.737 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:27.736+0000 7fb591c6fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:27.739 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:27.738+0000 7fb591c6fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:27.740 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:27.739+0000 7fb591c6fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:28.116 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 4 2020-09-12T14:25:28.116 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:25:28.116 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=4 2020-09-12T14:25:28.116 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:25:28.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:25:28.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:25:28.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:25:28.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:25:28.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.4 up' 2020-09-12T14:25:28.117 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:25:28.256 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:28.254+0000 7fb591c6fec0 -1 Falling back to public interface 2020-09-12T14:25:28.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:25:29.046 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:29.045+0000 7fb591c6fec0 -1 osd.4 0 log_to_monitors {default=true} 2020-09-12T14:25:29.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:25:29.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:25:29.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:25:29.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:25:29.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.4 up' 2020-09-12T14:25:29.521 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:25:29.981 INFO:tasks.workunit.client.0.smithi027.stdout:osd.4 up in weight 1 up_from 25 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/138360,v1:127.0.0.1:6835/138360] [v2:127.0.0.1:6836/138360,v1:127.0.0.1:6837/138360] exists,up 02cecd8c-7673-4267-859f-88af9093e38a 2020-09-12T14:25:29.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:25:29.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:25:29.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:25:29.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:54: _common_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:25:29.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:56: _common_test: run_osd td/osd-backfill-recovery-log 5 2020-09-12T14:25:29.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-recovery-log 2020-09-12T14:25:29.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:25:29.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=5 2020-09-12T14:25:29.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:25:29.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-recovery-log/5 2020-09-12T14:25:29.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 ' 2020-09-12T14:25:29.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:25:29.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:25:29.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:25:29.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-recovery-log/5' 2020-09-12T14:25:29.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-recovery-log/5/journal' 2020-09-12T14:25:29.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:25:29.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd_min_pg_log_entries=150 --osd_max_pg_log_entries=150 --osd_pg_log_dups_tracked=3000 --osd_pg_log_trim_min=10' 2020-09-12T14:25:29.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-recovery-log' 2020-09-12T14:25:29.984 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:25:29.984 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:25:29.984 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:25:29.984 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:25:29.985 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:25:29.985 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:25:29.985 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:25:29.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' 2020-09-12T14:25:29.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:25:29.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:25:29.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:25:29.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-recovery-log/$name.log' 2020-09-12T14:25:29.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-recovery-log/$name.pid' 2020-09-12T14:25:29.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:25:29.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:25:29.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:25:29.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:25:29.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:25:29.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-recovery-log/5 2020-09-12T14:25:29.987 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:25:29.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=ae3169a0-3c6d-44ee-89e2-5ea022baba17 2020-09-12T14:25:29.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd5 ae3169a0-3c6d-44ee-89e2-5ea022baba17' 2020-09-12T14:25:29.988 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:25:29.988 INFO:tasks.workunit.client.0.smithi027.stdout:add osd5 ae3169a0-3c6d-44ee-89e2-5ea022baba17 2020-09-12T14:25:29.997 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQBZ2lxfxqxtOxAAWEfSAJ3iHPj1GWpaCGXM4g== 2020-09-12T14:25:29.997 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQBZ2lxfxqxtOxAAWEfSAJ3iHPj1GWpaCGXM4g=="}' 2020-09-12T14:25:29.997 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new ae3169a0-3c6d-44ee-89e2-5ea022baba17 -i td/osd-backfill-recovery-log/5/new.json 2020-09-12T14:25:30.537 INFO:tasks.workunit.client.0.smithi027.stdout:5 2020-09-12T14:25:30.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-recovery-log/5/new.json 2020-09-12T14:25:30.550 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 5 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/5 --osd-journal=td/osd-backfill-recovery-log/5/journal --chdir= --osd_min_pg_log_entries=150 --osd_max_pg_log_entries=150 --osd_pg_log_dups_tracked=3000 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQBZ2lxfxqxtOxAAWEfSAJ3iHPj1GWpaCGXM4g== --osd-uuid ae3169a0-3c6d-44ee-89e2-5ea022baba17 2020-09-12T14:25:30.576 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:30.575+0000 7f0767159ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:30.578 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:30.577+0000 7f0767159ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:30.578 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:30.577+0000 7f0767159ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:30.579 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:30.578+0000 7f0767159ec0 -1 bluestore(td/osd-backfill-recovery-log/5/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/5/block: (2) No such file or directory 2020-09-12T14:25:30.579 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:30.578+0000 7f0767159ec0 -1 bluestore(td/osd-backfill-recovery-log/5/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/5/block: (2) No such file or directory 2020-09-12T14:25:30.579 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:30.578+0000 7f0767159ec0 -1 bluestore(td/osd-backfill-recovery-log/5/block) _read_bdev_label failed to open td/osd-backfill-recovery-log/5/block: (2) No such file or directory 2020-09-12T14:25:30.639 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:30.638+0000 7f0767159ec0 -1 bluestore(td/osd-backfill-recovery-log/5) _read_fsid unparsable uuid 2020-09-12T14:25:34.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-recovery-log/5/keyring 2020-09-12T14:25:34.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:25:34.938 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd5 key to auth repository 2020-09-12T14:25:34.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd5 key to auth repository 2020-09-12T14:25:34.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-recovery-log/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:25:35.500 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.5 2020-09-12T14:25:35.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.5 2020-09-12T14:25:35.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 5 --fsid=43b4d290-6eaa-4e6f-8289-41492cb5d476 --auth-supported=none --mon-host=127.0.0.1:7129 --osd_max_backfills=1 --debug_reserver=20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-recovery-log/5 --osd-journal=td/osd-backfill-recovery-log/5/journal --chdir= --osd_min_pg_log_entries=150 --osd_max_pg_log_entries=150 --osd_pg_log_dups_tracked=3000 --osd_pg_log_trim_min=10 --run-dir=td/osd-backfill-recovery-log '--admin-socket=/tmp/ceph-asok.98804/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-recovery-log/$name.log' '--pid-file=td/osd-backfill-recovery-log/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:25:35.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:25:35.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:25:35.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:25:35.525 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:35.524+0000 7f684ece6ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:35.531 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:35.531+0000 7f684ece6ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:35.532 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:35.532+0000 7f684ece6ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:25:35.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 5 2020-09-12T14:25:35.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:25:35.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=5 2020-09-12T14:25:35.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:25:35.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:25:35.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:25:35.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:25:35.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:25:35.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.5 up' 2020-09-12T14:25:35.927 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:25:36.060 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:36.059+0000 7f684ece6ec0 -1 Falling back to public interface 2020-09-12T14:25:36.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:25:36.858 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:25:36.858+0000 7f684ece6ec0 -1 osd.5 0 log_to_monitors {default=true} 2020-09-12T14:25:37.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:25:37.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:25:37.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:25:37.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:25:37.336 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:25:37.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.5 up' 2020-09-12T14:25:37.773 INFO:tasks.workunit.client.0.smithi027.stdout:osd.5 up in weight 1 up_from 30 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/139354,v1:127.0.0.1:6843/139354] [v2:127.0.0.1:6844/139354,v1:127.0.0.1:6845/139354] exists,up ae3169a0-3c6d-44ee-89e2-5ea022baba17 2020-09-12T14:25:37.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:25:37.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:25:37.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:25:37.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:59: _common_test: create_pool test 1 1 2020-09-12T14:25:37.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test 1 1 2020-09-12T14:25:38.349 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test' already exists 2020-09-12T14:25:38.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:25:39.360 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: seq 1 141 2020-09-12T14:25:39.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:39.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-1 /etc/passwd 2020-09-12T14:25:39.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:39.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-2 /etc/passwd 2020-09-12T14:25:39.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:39.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-3 /etc/passwd 2020-09-12T14:25:40.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:40.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-4 /etc/passwd 2020-09-12T14:25:40.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:40.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-5 /etc/passwd 2020-09-12T14:25:40.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:40.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-6 /etc/passwd 2020-09-12T14:25:40.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:40.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-7 /etc/passwd 2020-09-12T14:25:40.598 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:40.598 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-8 /etc/passwd 2020-09-12T14:25:40.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:40.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-9 /etc/passwd 2020-09-12T14:25:40.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:40.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-10 /etc/passwd 2020-09-12T14:25:41.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:41.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-11 /etc/passwd 2020-09-12T14:25:41.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:41.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-12 /etc/passwd 2020-09-12T14:25:41.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:41.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-13 /etc/passwd 2020-09-12T14:25:41.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:41.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-14 /etc/passwd 2020-09-12T14:25:41.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:41.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-15 /etc/passwd 2020-09-12T14:25:41.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:41.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-16 /etc/passwd 2020-09-12T14:25:41.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:41.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-17 /etc/passwd 2020-09-12T14:25:41.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:41.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-18 /etc/passwd 2020-09-12T14:25:42.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:42.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-19 /etc/passwd 2020-09-12T14:25:42.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:42.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-20 /etc/passwd 2020-09-12T14:25:42.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:42.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-21 /etc/passwd 2020-09-12T14:25:42.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:42.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-22 /etc/passwd 2020-09-12T14:25:42.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:42.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-23 /etc/passwd 2020-09-12T14:25:42.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:42.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-24 /etc/passwd 2020-09-12T14:25:43.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:43.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-25 /etc/passwd 2020-09-12T14:25:43.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:43.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-26 /etc/passwd 2020-09-12T14:25:43.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:43.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-27 /etc/passwd 2020-09-12T14:25:43.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:43.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-28 /etc/passwd 2020-09-12T14:25:43.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:43.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-29 /etc/passwd 2020-09-12T14:25:43.666 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:43.667 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-30 /etc/passwd 2020-09-12T14:25:43.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:43.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-31 /etc/passwd 2020-09-12T14:25:43.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:43.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-32 /etc/passwd 2020-09-12T14:25:44.094 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:44.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-33 /etc/passwd 2020-09-12T14:25:44.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:44.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-34 /etc/passwd 2020-09-12T14:25:44.371 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:44.371 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-35 /etc/passwd 2020-09-12T14:25:44.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:44.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-36 /etc/passwd 2020-09-12T14:25:44.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:44.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-37 /etc/passwd 2020-09-12T14:25:44.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:44.782 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-38 /etc/passwd 2020-09-12T14:25:44.915 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:44.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-39 /etc/passwd 2020-09-12T14:25:45.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:45.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-40 /etc/passwd 2020-09-12T14:25:45.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:45.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-41 /etc/passwd 2020-09-12T14:25:45.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:45.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-42 /etc/passwd 2020-09-12T14:25:45.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:45.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-43 /etc/passwd 2020-09-12T14:25:45.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:45.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-44 /etc/passwd 2020-09-12T14:25:45.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:45.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-45 /etc/passwd 2020-09-12T14:25:45.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:45.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-46 /etc/passwd 2020-09-12T14:25:45.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:45.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-47 /etc/passwd 2020-09-12T14:25:46.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:46.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-48 /etc/passwd 2020-09-12T14:25:46.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:46.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-49 /etc/passwd 2020-09-12T14:25:46.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:46.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-50 /etc/passwd 2020-09-12T14:25:46.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:46.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-51 /etc/passwd 2020-09-12T14:25:46.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:46.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-52 /etc/passwd 2020-09-12T14:25:46.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:46.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-53 /etc/passwd 2020-09-12T14:25:46.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:46.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-54 /etc/passwd 2020-09-12T14:25:46.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:46.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-55 /etc/passwd 2020-09-12T14:25:47.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:47.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-56 /etc/passwd 2020-09-12T14:25:47.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:47.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-57 /etc/passwd 2020-09-12T14:25:47.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:47.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-58 /etc/passwd 2020-09-12T14:25:47.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:47.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-59 /etc/passwd 2020-09-12T14:25:47.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:47.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-60 /etc/passwd 2020-09-12T14:25:47.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:47.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-61 /etc/passwd 2020-09-12T14:25:48.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:48.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-62 /etc/passwd 2020-09-12T14:25:48.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:48.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-63 /etc/passwd 2020-09-12T14:25:48.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:48.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-64 /etc/passwd 2020-09-12T14:25:48.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:48.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-65 /etc/passwd 2020-09-12T14:25:48.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:48.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-66 /etc/passwd 2020-09-12T14:25:48.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:48.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-67 /etc/passwd 2020-09-12T14:25:48.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:48.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-68 /etc/passwd 2020-09-12T14:25:48.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:48.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-69 /etc/passwd 2020-09-12T14:25:49.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:49.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-70 /etc/passwd 2020-09-12T14:25:49.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:49.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-71 /etc/passwd 2020-09-12T14:25:49.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:49.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-72 /etc/passwd 2020-09-12T14:25:49.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:49.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-73 /etc/passwd 2020-09-12T14:25:49.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:49.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-74 /etc/passwd 2020-09-12T14:25:49.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:49.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-75 /etc/passwd 2020-09-12T14:25:49.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:49.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-76 /etc/passwd 2020-09-12T14:25:50.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:50.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-77 /etc/passwd 2020-09-12T14:25:50.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:50.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-78 /etc/passwd 2020-09-12T14:25:50.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:50.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-79 /etc/passwd 2020-09-12T14:25:50.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:50.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-80 /etc/passwd 2020-09-12T14:25:50.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:50.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-81 /etc/passwd 2020-09-12T14:25:50.667 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:50.667 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-82 /etc/passwd 2020-09-12T14:25:50.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:50.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-83 /etc/passwd 2020-09-12T14:25:50.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:50.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-84 /etc/passwd 2020-09-12T14:25:51.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:51.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-85 /etc/passwd 2020-09-12T14:25:51.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:51.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-86 /etc/passwd 2020-09-12T14:25:51.262 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:51.262 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-87 /etc/passwd 2020-09-12T14:25:51.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:51.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-88 /etc/passwd 2020-09-12T14:25:51.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:51.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-89 /etc/passwd 2020-09-12T14:25:51.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:51.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-90 /etc/passwd 2020-09-12T14:25:51.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:51.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-91 /etc/passwd 2020-09-12T14:25:51.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:51.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-92 /etc/passwd 2020-09-12T14:25:52.084 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:52.084 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-93 /etc/passwd 2020-09-12T14:25:52.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:52.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-94 /etc/passwd 2020-09-12T14:25:52.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:52.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-95 /etc/passwd 2020-09-12T14:25:52.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:52.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-96 /etc/passwd 2020-09-12T14:25:52.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:52.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-97 /etc/passwd 2020-09-12T14:25:52.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:52.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-98 /etc/passwd 2020-09-12T14:25:52.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:52.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-99 /etc/passwd 2020-09-12T14:25:52.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:52.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-100 /etc/passwd 2020-09-12T14:25:53.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:53.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-101 /etc/passwd 2020-09-12T14:25:53.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:53.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-102 /etc/passwd 2020-09-12T14:25:53.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:53.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-103 /etc/passwd 2020-09-12T14:25:53.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:53.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-104 /etc/passwd 2020-09-12T14:25:53.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:53.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-105 /etc/passwd 2020-09-12T14:25:53.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:53.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-106 /etc/passwd 2020-09-12T14:25:53.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:53.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-107 /etc/passwd 2020-09-12T14:25:53.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:53.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-108 /etc/passwd 2020-09-12T14:25:54.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:54.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-109 /etc/passwd 2020-09-12T14:25:54.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:54.240 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-110 /etc/passwd 2020-09-12T14:25:54.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:54.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-111 /etc/passwd 2020-09-12T14:25:54.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:54.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-112 /etc/passwd 2020-09-12T14:25:54.625 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:54.625 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-113 /etc/passwd 2020-09-12T14:25:54.767 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:54.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-114 /etc/passwd 2020-09-12T14:25:54.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:54.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-115 /etc/passwd 2020-09-12T14:25:55.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:55.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-116 /etc/passwd 2020-09-12T14:25:55.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:55.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-117 /etc/passwd 2020-09-12T14:25:55.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:55.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-118 /etc/passwd 2020-09-12T14:25:55.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:55.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-119 /etc/passwd 2020-09-12T14:25:55.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:55.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-120 /etc/passwd 2020-09-12T14:25:55.673 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:55.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-121 /etc/passwd 2020-09-12T14:25:55.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:55.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-122 /etc/passwd 2020-09-12T14:25:55.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:55.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-123 /etc/passwd 2020-09-12T14:25:56.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:56.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-124 /etc/passwd 2020-09-12T14:25:56.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:56.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-125 /etc/passwd 2020-09-12T14:25:56.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:56.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-126 /etc/passwd 2020-09-12T14:25:56.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:56.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-127 /etc/passwd 2020-09-12T14:25:56.612 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:56.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-128 /etc/passwd 2020-09-12T14:25:56.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:56.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-129 /etc/passwd 2020-09-12T14:25:56.864 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:56.864 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-130 /etc/passwd 2020-09-12T14:25:56.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:56.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-131 /etc/passwd 2020-09-12T14:25:57.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:57.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-132 /etc/passwd 2020-09-12T14:25:57.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:57.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-133 /etc/passwd 2020-09-12T14:25:57.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:57.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-134 /etc/passwd 2020-09-12T14:25:57.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:57.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-135 /etc/passwd 2020-09-12T14:25:57.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:57.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-136 /etc/passwd 2020-09-12T14:25:57.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:57.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-137 /etc/passwd 2020-09-12T14:25:57.997 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:57.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-138 /etc/passwd 2020-09-12T14:25:58.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:58.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-139 /etc/passwd 2020-09-12T14:25:58.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:58.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-140 /etc/passwd 2020-09-12T14:25:58.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:61: _common_test: for j in $(seq 1 $objects) 2020-09-12T14:25:58.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:63: _common_test: rados -p test put obj-141 /etc/passwd 2020-09-12T14:25:58.510 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:67: _common_test: ceph pg dump pgs --format=json 2020-09-12T14:25:58.510 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:67: _common_test: jq '.pg_stats[0].up[]' 2020-09-12T14:25:58.905 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:25:58.914 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:67: _common_test: ceph osd out 1 0 2 2020-09-12T14:25:59.454 INFO:tasks.workunit.client.0.smithi027.stderr:osd.1 is already out. osd.0 is already out. osd.2 is already out. 2020-09-12T14:25:59.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:68: _common_test: '[' 20 '!=' 0 ']' 2020-09-12T14:25:59.464 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:69: _common_test: seq 1 20 2020-09-12T14:25:59.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:69: _common_test: for j in $(seq 1 $moreobjects) 2020-09-12T14:25:59.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:71: _common_test: rados -p test put obj-more-1 /etc/passwd 2020-09-12T14:26:03.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:69: _common_test: for j in $(seq 1 $moreobjects) 2020-09-12T14:26:03.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:71: _common_test: rados -p test put obj-more-2 /etc/passwd 2020-09-12T14:26:03.767 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:69: _common_test: for j in $(seq 1 $moreobjects) 2020-09-12T14:26:03.767 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:71: _common_test: rados -p test put obj-more-3 /etc/passwd 2020-09-12T14:26:04.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:69: _common_test: for j in $(seq 1 $moreobjects) 2020-09-12T14:26:04.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:71: _common_test: rados -p test put obj-more-4 /etc/passwd 2020-09-12T14:26:04.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:69: _common_test: for j in $(seq 1 $moreobjects) 2020-09-12T14:26:04.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:71: _common_test: rados -p test put obj-more-5 /etc/passwd 2020-09-12T14:26:04.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:69: _common_test: for j in $(seq 1 $moreobjects) 2020-09-12T14:26:04.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:71: _common_test: rados -p test put obj-more-6 /etc/passwd 2020-09-12T14:26:04.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:69: _common_test: for j in $(seq 1 $moreobjects) 2020-09-12T14:26:04.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:71: _common_test: rados -p test put obj-more-7 /etc/passwd 2020-09-12T14:26:05.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:69: _common_test: for j in $(seq 1 $moreobjects) 2020-09-12T14:26:05.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:71: _common_test: rados -p test put obj-more-8 /etc/passwd 2020-09-12T14:26:05.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:69: _common_test: for j in $(seq 1 $moreobjects) 2020-09-12T14:26:05.378 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:71: _common_test: rados -p test put obj-more-9 /etc/passwd 2020-09-12T14:26:05.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:69: _common_test: for j in $(seq 1 $moreobjects) 2020-09-12T14:26:05.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:71: _common_test: rados -p test put obj-more-10 /etc/passwd 2020-09-12T14:26:05.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:69: _common_test: for j in $(seq 1 $moreobjects) 2020-09-12T14:26:05.823 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:71: _common_test: rados -p test put obj-more-11 /etc/passwd 2020-09-12T14:26:06.083 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:69: _common_test: for j in $(seq 1 $moreobjects) 2020-09-12T14:26:06.083 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:71: _common_test: rados -p test put obj-more-12 /etc/passwd 2020-09-12T14:26:06.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:69: _common_test: for j in $(seq 1 $moreobjects) 2020-09-12T14:26:06.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:71: _common_test: rados -p test put obj-more-13 /etc/passwd 2020-09-12T14:26:06.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:69: _common_test: for j in $(seq 1 $moreobjects) 2020-09-12T14:26:06.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:71: _common_test: rados -p test put obj-more-14 /etc/passwd 2020-09-12T14:26:06.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:69: _common_test: for j in $(seq 1 $moreobjects) 2020-09-12T14:26:06.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:71: _common_test: rados -p test put obj-more-15 /etc/passwd 2020-09-12T14:26:07.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:69: _common_test: for j in $(seq 1 $moreobjects) 2020-09-12T14:26:07.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:71: _common_test: rados -p test put obj-more-16 /etc/passwd 2020-09-12T14:26:07.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:69: _common_test: for j in $(seq 1 $moreobjects) 2020-09-12T14:26:07.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:71: _common_test: rados -p test put obj-more-17 /etc/passwd 2020-09-12T14:26:07.551 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:69: _common_test: for j in $(seq 1 $moreobjects) 2020-09-12T14:26:07.552 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:71: _common_test: rados -p test put obj-more-18 /etc/passwd 2020-09-12T14:26:07.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:69: _common_test: for j in $(seq 1 $moreobjects) 2020-09-12T14:26:07.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:71: _common_test: rados -p test put obj-more-19 /etc/passwd 2020-09-12T14:26:08.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:69: _common_test: for j in $(seq 1 $moreobjects) 2020-09-12T14:26:08.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:71: _common_test: rados -p test put obj-more-20 /etc/passwd 2020-09-12T14:26:08.316 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:74: _common_test: sleep 1 2020-09-12T14:26:09.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:75: _common_test: wait_for_clean 2020-09-12T14:26:09.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:26:09.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:26:09.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:26:09.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:26:09.319 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 90 .1 2020-09-12T14:26:09.319 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:26:09.319 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:26:09.320 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:26:09.320 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:26:09.320 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:26:09.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:26:09.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:26:09.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:26:09.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:26:09.426 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:26:09.841 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:26:09.841 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:26:09.841 INFO:tasks.workunit.client.0.smithi027.stderr:2 2020-09-12T14:26:09.841 INFO:tasks.workunit.client.0.smithi027.stderr:3 2020-09-12T14:26:09.842 INFO:tasks.workunit.client.0.smithi027.stderr:4 2020-09-12T14:26:09.842 INFO:tasks.workunit.client.0.smithi027.stderr:5' 2020-09-12T14:26:09.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:26:09.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:26:09.842 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:26:10.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836498 2020-09-12T14:26:10.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836498 2020-09-12T14:26:10.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836498' 2020-09-12T14:26:10.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:26:10.005 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:26:10.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672976 2020-09-12T14:26:10.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672976 2020-09-12T14:26:10.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836498 1-42949672976' 2020-09-12T14:26:10.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:26:10.146 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:26:10.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509454 2020-09-12T14:26:10.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509454 2020-09-12T14:26:10.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836498 1-42949672976 2-64424509454' 2020-09-12T14:26:10.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:26:10.287 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2020-09-12T14:26:10.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=85899345932 2020-09-12T14:26:10.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 85899345932 2020-09-12T14:26:10.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836498 1-42949672976 2-64424509454 3-85899345932' 2020-09-12T14:26:10.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:26:10.428 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2020-09-12T14:26:10.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=107374182411 2020-09-12T14:26:10.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 107374182411 2020-09-12T14:26:10.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836498 1-42949672976 2-64424509454 3-85899345932 4-107374182411' 2020-09-12T14:26:10.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:26:10.568 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2020-09-12T14:26:10.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=128849018889 2020-09-12T14:26:10.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 128849018889 2020-09-12T14:26:10.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836498 1-42949672976 2-64424509454 3-85899345932 4-107374182411 5-128849018889' 2020-09-12T14:26:10.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:26:10.710 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836498 2020-09-12T14:26:10.710 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:26:10.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:26:10.711 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836498 2020-09-12T14:26:10.711 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:26:10.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836498 2020-09-12T14:26:10.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836498' 2020-09-12T14:26:10.712 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:26:10.713 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836498 2020-09-12T14:26:11.113 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836496 -lt 21474836498 2020-09-12T14:26:11.113 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:26:12.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:26:12.115 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:26:12.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836498 -lt 21474836498 2020-09-12T14:26:12.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:26:12.536 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672976 2020-09-12T14:26:12.537 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:26:12.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:26:12.537 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672976 2020-09-12T14:26:12.538 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:26:12.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672976 2020-09-12T14:26:12.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672976' 2020-09-12T14:26:12.539 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672976 2020-09-12T14:26:12.539 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:26:13.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672976 -lt 42949672976 2020-09-12T14:26:13.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:26:13.003 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509454 2020-09-12T14:26:13.003 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:26:13.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:26:13.004 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509454 2020-09-12T14:26:13.004 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:26:13.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509454 2020-09-12T14:26:13.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509454' 2020-09-12T14:26:13.005 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509454 2020-09-12T14:26:13.005 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:26:13.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509454 -lt 64424509454 2020-09-12T14:26:13.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:26:13.411 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 3-85899345932 2020-09-12T14:26:13.411 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:26:13.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=3 2020-09-12T14:26:13.412 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 3-85899345932 2020-09-12T14:26:13.412 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:26:13.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=85899345932 2020-09-12T14:26:13.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.3 seq 85899345932' 2020-09-12T14:26:13.413 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.3 seq 85899345932 2020-09-12T14:26:13.414 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 3 2020-09-12T14:26:13.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 85899345933 -lt 85899345932 2020-09-12T14:26:13.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:26:13.826 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 4-107374182411 2020-09-12T14:26:13.826 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:26:13.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=4 2020-09-12T14:26:13.827 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 4-107374182411 2020-09-12T14:26:13.827 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:26:13.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=107374182411 2020-09-12T14:26:13.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.4 seq 107374182411' 2020-09-12T14:26:13.828 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.4 seq 107374182411 2020-09-12T14:26:13.829 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 4 2020-09-12T14:26:14.231 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 107374182411 -lt 107374182411 2020-09-12T14:26:14.231 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:26:14.232 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 5-128849018889 2020-09-12T14:26:14.232 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:26:14.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=5 2020-09-12T14:26:14.233 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 5-128849018889 2020-09-12T14:26:14.233 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:26:14.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=128849018889 2020-09-12T14:26:14.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.5 seq 128849018889' 2020-09-12T14:26:14.234 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.5 seq 128849018889 2020-09-12T14:26:14.234 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 5 2020-09-12T14:26:14.663 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 128849018890 -lt 128849018889 2020-09-12T14:26:14.664 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:26:14.664 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:26:14.664 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:26:15.236 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 1 == 0 2020-09-12T14:26:15.236 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:26:15.236 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:26:15.236 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:26:15.236 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:26:15.237 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:26:15.237 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:26:15.237 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:26:15.662 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:26:15.663 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:26:15.663 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:26:15.663 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:26:16.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:26:16.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' -1 2020-09-12T14:26:16.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1598: wait_for_clean: loop=0 2020-09-12T14:26:16.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1599: wait_for_clean: num_active_clean=0 2020-09-12T14:26:16.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:26:16.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:26:16.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:26:16.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:26:16.304 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:26:16.304 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:26:16.304 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:26:16.304 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:26:16.305 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:26:16.305 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:26:16.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:26:16.695 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:26:16.695 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:26:16.696 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:26:17.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:26:17.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:26:17.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:26:17.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:26:17.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:26:17.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:26:17.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:26:17.229 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:26:17.229 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:26:17.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=27696 2020-09-12T14:26:17.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 27696 '!=' null 2020-09-12T14:26:17.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:26:17.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:26:17.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:26:17.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:26:17.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:26:17.859 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:26:17.859 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:26:17.859 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:26:17.859 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:26:17.859 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:26:17.860 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:26:18.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:26:18.250 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:26:18.250 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:26:18.250 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:26:18.782 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:26:18.782 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:26:18.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:26:18.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:26:18.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:26:18.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:26:18.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:26:18.783 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:26:18.783 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:26:19.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=27696 2020-09-12T14:26:19.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 27696 '!=' null 2020-09-12T14:26:19.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:26:19.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:26:19.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:26:19.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:26:19.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:26:19.410 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:26:19.410 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:26:19.410 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:26:19.410 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:26:19.411 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:26:19.411 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:26:19.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:26:19.812 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:26:19.812 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:26:19.812 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:26:20.371 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:26:20.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:26:20.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:26:20.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:26:20.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:26:20.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:26:20.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:26:20.372 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:26:20.373 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:26:20.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=40580 2020-09-12T14:26:20.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 40580 '!=' null 2020-09-12T14:26:20.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:26:20.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:26:20.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:26:21.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:26:21.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:26:21.020 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:26:21.020 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:26:21.020 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:26:21.021 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:26:21.021 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:26:21.021 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:26:21.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:26:21.414 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:26:21.415 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:26:21.415 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:26:21.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:26:21.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:26:21.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:26:21.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:26:21.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:26:21.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:26:21.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:26:21.991 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:26:21.991 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:26:22.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=27053 2020-09-12T14:26:22.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 27053 '!=' null 2020-09-12T14:26:22.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:26:22.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:26:22.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:26:22.622 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:26:22.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:26:22.623 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:26:22.624 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:26:22.624 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:26:22.624 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:26:22.624 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:26:22.624 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:26:23.014 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:26:23.015 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:26:23.015 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:26:23.015 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:26:23.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:26:23.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:26:23.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:26:23.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:26:23.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:26:23.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:26:23.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:26:23.546 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:26:23.546 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:26:24.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=25121 2020-09-12T14:26:24.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 25121 '!=' null 2020-09-12T14:26:24.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:26:24.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:26:24.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:26:24.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:26:24.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:26:24.257 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:26:24.257 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:26:24.257 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:26:24.257 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:26:24.258 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:26:24.258 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:26:24.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:26:24.645 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:26:24.646 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:26:24.646 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:26:25.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:26:25.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:26:25.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:26:25.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:26:25.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:26:25.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:26:25.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:26:25.183 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:26:25.184 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:26:25.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=38648 2020-09-12T14:26:25.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 38648 '!=' null 2020-09-12T14:26:25.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:26:25.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:26:25.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:26:25.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:26:25.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:26:25.837 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:26:25.837 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:26:25.838 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:26:25.838 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:26:25.838 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:26:25.838 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:26:26.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:26:26.234 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:26:26.234 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:26:26.235 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:26:26.762 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:26:26.762 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:26:26.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:26:26.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:26:26.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:26:26.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:26:26.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:26:26.763 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:26:26.764 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:26:27.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=38648 2020-09-12T14:26:27.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 38648 '!=' null 2020-09-12T14:26:27.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:26:27.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:26:27.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:26:27.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:26:27.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:26:27.396 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:26:27.396 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:26:27.397 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:26:27.397 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:26:27.397 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:26:27.397 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:26:27.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:26:27.786 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:26:27.786 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:26:27.787 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:26:28.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:26:28.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:26:28.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:26:28.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:26:28.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:26:28.353 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:26:28.353 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:26:28.353 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:26:28.353 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:26:28.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=26409 2020-09-12T14:26:28.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 26409 '!=' null 2020-09-12T14:26:28.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:26:28.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:26:28.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:26:28.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:26:28.997 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:26:28.997 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:26:28.998 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:26:28.998 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:26:28.998 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:26:28.998 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:26:28.998 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:26:29.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:26:29.393 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:26:29.393 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:26:29.393 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:26:29.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:26:29.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:26:29.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:26:29.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:26:29.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:26:29.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:26:29.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:26:29.919 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:26:29.919 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:26:30.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=40581 2020-09-12T14:26:30.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 40581 '!=' null 2020-09-12T14:26:30.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:26:30.466 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:26:30.466 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:26:30.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:26:30.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:26:30.567 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:26:30.567 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:26:30.567 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:26:30.567 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:26:30.567 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:26:30.568 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:26:30.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:26:30.964 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:26:30.964 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:26:30.965 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:26:31.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:26:31.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:26:31.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:26:31.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:26:31.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:26:31.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:26:31.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:26:31.514 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:26:31.514 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:26:32.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=27698 2020-09-12T14:26:32.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 27698 '!=' null 2020-09-12T14:26:32.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:26:32.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:26:32.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:26:32.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:26:32.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:26:32.190 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:26:32.190 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:26:32.190 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:26:32.190 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:26:32.191 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:26:32.191 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:26:32.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:26:32.588 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:26:32.588 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:26:32.589 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:26:33.119 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:26:33.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:26:33.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:26:33.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:26:33.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:26:33.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:26:33.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:26:33.121 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:26:33.121 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:26:33.662 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=27698 2020-09-12T14:26:33.663 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 27698 '!=' null 2020-09-12T14:26:33.663 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:26:33.663 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:26:33.663 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:26:33.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:26:33.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:26:33.764 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:26:33.764 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:26:33.765 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:26:33.765 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:26:33.765 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:26:33.765 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:26:34.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:26:34.157 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:26:34.157 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:26:34.158 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:26:34.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:26:34.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:26:34.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:26:34.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:26:34.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:26:34.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:26:34.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:26:34.692 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:26:34.692 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:26:35.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=27697 2020-09-12T14:26:35.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 27697 '!=' null 2020-09-12T14:26:35.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:26:35.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:26:35.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:26:35.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:26:35.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:26:35.327 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:26:35.327 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:26:35.327 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:26:35.327 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:26:35.328 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:26:35.328 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:26:35.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:26:35.726 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:26:35.726 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:26:35.726 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:26:36.262 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:26:36.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:26:36.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:26:36.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:26:36.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:26:36.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:26:36.264 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:26:36.264 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:26:36.264 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:26:36.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=41869 2020-09-12T14:26:36.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 41869 '!=' null 2020-09-12T14:26:36.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:26:36.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:26:36.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:26:36.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:26:36.928 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:26:36.928 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:26:36.928 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:26:36.928 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:26:36.928 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:26:36.929 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:26:36.929 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:26:37.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:26:37.325 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:26:37.325 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:26:37.325 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:26:37.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:26:37.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:26:37.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:26:37.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:26:37.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:26:37.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:26:37.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:26:37.858 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:26:37.858 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:26:38.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=21256 2020-09-12T14:26:38.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 21256 '!=' null 2020-09-12T14:26:38.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:26:38.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:26:38.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:26:38.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:26:38.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:26:38.490 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:26:38.490 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:26:38.490 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:26:38.490 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:26:38.491 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:26:38.491 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:26:38.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:26:38.878 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:26:38.879 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:26:38.879 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:26:39.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:26:39.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:26:39.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:26:39.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:26:39.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:26:39.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:26:39.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:26:39.429 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:26:39.429 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:26:39.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=21255 2020-09-12T14:26:39.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 21255 '!=' null 2020-09-12T14:26:39.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:26:39.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:26:39.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:26:40.062 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:26:40.062 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:26:40.063 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:26:40.063 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:26:40.063 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:26:40.064 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:26:40.064 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:26:40.064 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:26:40.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:26:40.458 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:26:40.459 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:26:40.459 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:26:40.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:26:40.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:26:40.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:26:40.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:26:40.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:26:40.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:26:40.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:26:40.986 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:26:40.986 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:26:41.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=21255 2020-09-12T14:26:41.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 21255 '!=' null 2020-09-12T14:26:41.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:26:41.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:26:41.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:26:41.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:26:41.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:26:41.619 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:26:41.619 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:26:41.619 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:26:41.619 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:26:41.619 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:26:41.620 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:26:42.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:26:42.004 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:26:42.005 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:26:42.005 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:26:42.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 1 2020-09-12T14:26:42.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T14:26:42.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T14:26:42.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:77: _common_test: flush_pg_stats 2020-09-12T14:26:42.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:26:42.533 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:26:42.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:26:42.937 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:26:42.937 INFO:tasks.workunit.client.0.smithi027.stderr:2 2020-09-12T14:26:42.937 INFO:tasks.workunit.client.0.smithi027.stderr:3 2020-09-12T14:26:42.937 INFO:tasks.workunit.client.0.smithi027.stderr:4 2020-09-12T14:26:42.937 INFO:tasks.workunit.client.0.smithi027.stderr:5' 2020-09-12T14:26:42.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:26:42.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:26:42.938 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:26:43.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836506 2020-09-12T14:26:43.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836506 2020-09-12T14:26:43.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836506' 2020-09-12T14:26:43.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:26:43.103 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:26:43.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672985 2020-09-12T14:26:43.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672985 2020-09-12T14:26:43.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836506 1-42949672985' 2020-09-12T14:26:43.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:26:43.243 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:26:43.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509463 2020-09-12T14:26:43.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509463 2020-09-12T14:26:43.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836506 1-42949672985 2-64424509463' 2020-09-12T14:26:43.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:26:43.383 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2020-09-12T14:26:43.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=85899345941 2020-09-12T14:26:43.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 85899345941 2020-09-12T14:26:43.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836506 1-42949672985 2-64424509463 3-85899345941' 2020-09-12T14:26:43.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:26:43.523 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2020-09-12T14:26:43.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=107374182419 2020-09-12T14:26:43.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 107374182419 2020-09-12T14:26:43.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836506 1-42949672985 2-64424509463 3-85899345941 4-107374182419' 2020-09-12T14:26:43.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:26:43.680 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2020-09-12T14:26:43.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=128849018898 2020-09-12T14:26:43.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 128849018898 2020-09-12T14:26:43.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836506 1-42949672985 2-64424509463 3-85899345941 4-107374182419 5-128849018898' 2020-09-12T14:26:43.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:26:43.819 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836506 2020-09-12T14:26:43.820 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:26:43.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:26:43.820 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836506 2020-09-12T14:26:43.820 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:26:43.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836506 2020-09-12T14:26:43.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836506' 2020-09-12T14:26:43.821 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:26:43.821 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836506 2020-09-12T14:26:44.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836506 -lt 21474836506 2020-09-12T14:26:44.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:26:44.224 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672985 2020-09-12T14:26:44.224 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:26:44.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:26:44.225 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672985 2020-09-12T14:26:44.225 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:26:44.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672985 2020-09-12T14:26:44.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672985' 2020-09-12T14:26:44.227 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672985 2020-09-12T14:26:44.227 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:26:44.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672985 -lt 42949672985 2020-09-12T14:26:44.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:26:44.641 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509463 2020-09-12T14:26:44.641 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:26:44.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:26:44.642 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509463 2020-09-12T14:26:44.642 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:26:44.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509463 2020-09-12T14:26:44.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509463' 2020-09-12T14:26:44.643 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509463 2020-09-12T14:26:44.644 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:26:45.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509463 -lt 64424509463 2020-09-12T14:26:45.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:26:45.061 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 3-85899345941 2020-09-12T14:26:45.061 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:26:45.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=3 2020-09-12T14:26:45.062 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 3-85899345941 2020-09-12T14:26:45.062 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:26:45.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=85899345941 2020-09-12T14:26:45.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.3 seq 85899345941' 2020-09-12T14:26:45.063 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.3 seq 85899345941 2020-09-12T14:26:45.063 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 3 2020-09-12T14:26:45.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 85899345941 -lt 85899345941 2020-09-12T14:26:45.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:26:45.492 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 4-107374182419 2020-09-12T14:26:45.492 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:26:45.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=4 2020-09-12T14:26:45.493 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 4-107374182419 2020-09-12T14:26:45.493 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:26:45.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=107374182419 2020-09-12T14:26:45.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.4 seq 107374182419' 2020-09-12T14:26:45.495 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.4 seq 107374182419 2020-09-12T14:26:45.495 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 4 2020-09-12T14:26:45.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 107374182420 -lt 107374182419 2020-09-12T14:26:45.894 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:26:45.894 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 5-128849018898 2020-09-12T14:26:45.894 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:26:45.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=5 2020-09-12T14:26:45.895 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 5-128849018898 2020-09-12T14:26:45.895 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:26:45.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=128849018898 2020-09-12T14:26:45.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.5 seq 128849018898' 2020-09-12T14:26:45.897 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.5 seq 128849018898 2020-09-12T14:26:45.897 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 5 2020-09-12T14:26:46.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 128849018898 -lt 128849018898 2020-09-12T14:26:46.326 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:79: _common_test: ceph pg dump pgs --format=json 2020-09-12T14:26:46.326 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:79: _common_test: jq '.pg_stats[0].up_primary' 2020-09-12T14:26:46.728 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:26:46.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:79: _common_test: newprimary=4 2020-09-12T14:26:46.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:80: _common_test: kill_daemons 2020-09-12T14:26:46.739 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:26:46.739 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:26:46.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:26:46.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:26:46.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:26:47.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:26:47.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:82: _common_test: ERRORS=0 2020-09-12T14:26:47.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:83: _common_test: _objectstore_tool_nodown td/osd-backfill-recovery-log 4 --no-mon-config --pgid 1.0 --op log 2020-09-12T14:26:47.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:83: _common_test: tee td/osd-backfill-recovery-log/result.log 2020-09-12T14:26:47.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1195: _objectstore_tool_nodown: local dir=td/osd-backfill-recovery-log 2020-09-12T14:26:47.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: _objectstore_tool_nodown: shift 2020-09-12T14:26:47.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: _objectstore_tool_nodown: local id=4 2020-09-12T14:26:47.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1198: _objectstore_tool_nodown: shift 2020-09-12T14:26:47.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: _objectstore_tool_nodown: local osd_data=td/osd-backfill-recovery-log/4 2020-09-12T14:26:47.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1202: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-backfill-recovery-log/4 --no-mon-config --pgid 1.0 --op log 2020-09-12T14:26:47.973 INFO:tasks.workunit.client.0.smithi027.stdout:{ 2020-09-12T14:26:47.973 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_log_t": { 2020-09-12T14:26:47.974 INFO:tasks.workunit.client.0.smithi027.stdout: "head": "38'161", 2020-09-12T14:26:47.974 INFO:tasks.workunit.client.0.smithi027.stdout: "tail": "34'10", 2020-09-12T14:26:47.974 INFO:tasks.workunit.client.0.smithi027.stdout: "log": [ 2020-09-12T14:26:47.974 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:47.974 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:47.974 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:1e6eca61:::obj-11:head", 2020-09-12T14:26:47.974 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'11", 2020-09-12T14:26:47.975 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:47.975 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4221.0:1", 2020-09-12T14:26:47.975 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:47.975 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:41.027072+0000", 2020-09-12T14:26:47.975 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:47.975 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:47.975 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:47.976 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:47.976 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:47.976 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:47.976 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:47.976 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:47.976 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:47.976 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:47.976 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:47.977 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:47.977 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:47.977 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:47.977 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:47.977 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:47.977 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:47.977 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:47.978 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:18dee445:::obj-12:head", 2020-09-12T14:26:47.978 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'12", 2020-09-12T14:26:47.978 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:47.978 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4224.0:1", 2020-09-12T14:26:47.978 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:47.978 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:41.162804+0000", 2020-09-12T14:26:47.978 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:47.979 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:47.979 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:47.979 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:47.979 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:47.979 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:47.979 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:47.979 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:47.979 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:47.980 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:47.980 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:47.980 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:47.980 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:47.980 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:47.980 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:47.980 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:47.980 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:47.981 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:47.981 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:283c5c90:::obj-13:head", 2020-09-12T14:26:47.981 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'13", 2020-09-12T14:26:47.981 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:47.981 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4227.0:1", 2020-09-12T14:26:47.981 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:47.981 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:41.354459+0000", 2020-09-12T14:26:47.982 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:47.982 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:47.982 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:47.982 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:47.982 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:47.982 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:47.982 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:47.982 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:47.983 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:47.983 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:47.983 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:47.983 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:47.983 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:47.983 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:47.983 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:47.984 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:47.984 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:47.984 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:47.984 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:cc896bf0:::obj-14:head", 2020-09-12T14:26:47.984 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'14", 2020-09-12T14:26:47.984 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:47.984 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4230.0:1", 2020-09-12T14:26:47.984 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:47.985 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:41.489070+0000", 2020-09-12T14:26:47.985 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:47.985 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:47.985 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:47.985 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:47.985 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:47.985 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:47.985 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:47.986 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:47.986 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:47.986 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:47.986 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:47.987 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:47.987 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:47.987 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:47.987 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:47.987 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:47.987 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:47.987 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:47.987 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:a581f807:::obj-15:head", 2020-09-12T14:26:47.988 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'15", 2020-09-12T14:26:47.988 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:47.988 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4233.0:1", 2020-09-12T14:26:47.988 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:47.988 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:41.631278+0000", 2020-09-12T14:26:47.988 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:47.988 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:47.989 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:47.989 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:47.989 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:47.989 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:47.989 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:47.989 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:47.989 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:47.989 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:47.990 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:47.990 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:47.990 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:47.990 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:47.990 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:47.990 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:47.990 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:47.991 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:47.991 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:c902ca7c:::obj-16:head", 2020-09-12T14:26:47.991 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'16", 2020-09-12T14:26:47.991 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:47.991 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4236.0:1", 2020-09-12T14:26:47.991 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:47.991 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:41.773593+0000", 2020-09-12T14:26:47.991 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:47.992 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:47.992 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:47.992 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:47.992 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:47.992 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:47.992 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:47.992 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:47.992 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:47.993 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:47.993 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:47.993 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:47.993 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:47.993 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:47.993 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:47.993 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:47.993 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:47.994 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:47.994 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:655283c8:::obj-17:head", 2020-09-12T14:26:47.994 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'17", 2020-09-12T14:26:47.994 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:47.994 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4239.0:1", 2020-09-12T14:26:47.994 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:47.994 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:41.916247+0000", 2020-09-12T14:26:47.995 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:47.995 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:47.995 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:47.995 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:47.995 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:47.995 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:47.995 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:47.995 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:47.996 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:47.996 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:47.996 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:47.996 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:47.996 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:47.996 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:47.996 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:47.996 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:47.997 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:47.997 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:47.997 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:d759d762:::obj-18:head", 2020-09-12T14:26:47.997 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'18", 2020-09-12T14:26:47.997 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:47.997 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4242.0:1", 2020-09-12T14:26:47.997 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:47.998 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:42.025573+0000", 2020-09-12T14:26:47.998 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:47.998 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:47.998 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:47.998 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:47.998 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:47.998 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:47.998 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:47.999 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:47.999 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:47.999 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:47.999 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:47.999 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:47.999 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:47.999 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.000 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.000 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.000 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.000 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.000 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:1b934a2d:::obj-19:head", 2020-09-12T14:26:48.000 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'19", 2020-09-12T14:26:48.000 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.000 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4245.0:1", 2020-09-12T14:26:48.001 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.001 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:42.159883+0000", 2020-09-12T14:26:48.001 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.001 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.001 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.001 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.001 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.001 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.002 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.002 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.002 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.002 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.002 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.002 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.002 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.003 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.003 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.003 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.003 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.003 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.003 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:1d0ea2ca:::obj-20:head", 2020-09-12T14:26:48.003 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'20", 2020-09-12T14:26:48.004 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.004 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4248.0:1", 2020-09-12T14:26:48.004 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.004 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:42.301953+0000", 2020-09-12T14:26:48.004 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.004 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.004 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.004 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.005 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.005 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.005 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.005 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.005 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.005 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.005 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.005 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.006 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.006 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.006 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.006 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.006 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.006 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.006 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:ab396690:::obj-21:head", 2020-09-12T14:26:48.007 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'21", 2020-09-12T14:26:48.007 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.007 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4251.0:1", 2020-09-12T14:26:48.007 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.007 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:42.435487+0000", 2020-09-12T14:26:48.007 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.007 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.007 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.008 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.008 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.008 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.008 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.008 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.008 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.008 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.008 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.009 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.009 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.009 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.009 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.009 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.009 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.009 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.009 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:254983db:::obj-22:head", 2020-09-12T14:26:48.010 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'22", 2020-09-12T14:26:48.010 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.010 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4254.0:1", 2020-09-12T14:26:48.010 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.010 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:42.636728+0000", 2020-09-12T14:26:48.010 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.010 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.011 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.011 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.011 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.011 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.011 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.011 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.011 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.011 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.012 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.012 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.012 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.012 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.012 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.012 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.012 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.013 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.013 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:e656299f:::obj-23:head", 2020-09-12T14:26:48.013 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'23", 2020-09-12T14:26:48.013 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.013 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4257.0:1", 2020-09-12T14:26:48.013 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.013 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:42.763288+0000", 2020-09-12T14:26:48.013 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.014 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.014 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.014 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.014 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.014 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.014 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.014 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.014 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.015 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.015 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.015 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.015 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.015 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.015 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.015 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.016 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.016 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.016 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:5cdf412b:::obj-24:head", 2020-09-12T14:26:48.016 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'24", 2020-09-12T14:26:48.016 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.016 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4260.0:1", 2020-09-12T14:26:48.016 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.016 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:42.924536+0000", 2020-09-12T14:26:48.017 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.017 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.017 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.017 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.017 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.017 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.017 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.017 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.018 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.018 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.018 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.018 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.018 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.018 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.018 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.019 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.019 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.019 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.019 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:cacae143:::obj-25:head", 2020-09-12T14:26:48.019 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'25", 2020-09-12T14:26:48.019 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.019 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4263.0:1", 2020-09-12T14:26:48.019 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.020 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:43.046862+0000", 2020-09-12T14:26:48.020 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.020 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.020 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.020 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.020 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.020 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.020 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.021 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.021 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.021 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.021 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.021 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.021 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.021 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.022 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.022 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.022 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.022 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.022 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:34b5bff6:::obj-26:head", 2020-09-12T14:26:48.022 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'26", 2020-09-12T14:26:48.022 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.022 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4266.0:1", 2020-09-12T14:26:48.023 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.023 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:43.165130+0000", 2020-09-12T14:26:48.023 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.023 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.023 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.023 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.023 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.023 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.024 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.024 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.024 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.024 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.024 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.024 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.024 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.025 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.025 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.025 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.025 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.025 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.025 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:a354f8f6:::obj-27:head", 2020-09-12T14:26:48.025 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'27", 2020-09-12T14:26:48.025 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.026 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4269.0:1", 2020-09-12T14:26:48.026 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.026 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:43.298643+0000", 2020-09-12T14:26:48.026 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.026 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.026 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.026 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.027 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.027 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.027 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.027 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.027 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.027 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.028 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.028 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.028 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.028 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.028 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.028 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.028 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.029 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.029 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:7421db49:::obj-28:head", 2020-09-12T14:26:48.029 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'28", 2020-09-12T14:26:48.029 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.029 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4272.0:1", 2020-09-12T14:26:48.029 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.029 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:43.425197+0000", 2020-09-12T14:26:48.030 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.030 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.030 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.030 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.030 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.030 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.030 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.030 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.031 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.031 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.031 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.031 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.031 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.031 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.031 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.032 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.032 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.032 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.032 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:54bc79ba:::obj-29:head", 2020-09-12T14:26:48.032 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'29", 2020-09-12T14:26:48.032 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.032 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4275.0:1", 2020-09-12T14:26:48.032 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.033 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:43.551759+0000", 2020-09-12T14:26:48.033 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.033 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.033 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.033 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.033 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.033 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.034 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.034 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.034 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.034 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.034 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.034 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.034 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.034 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.035 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.035 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.035 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.035 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.035 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:01b15595:::obj-30:head", 2020-09-12T14:26:48.035 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'30", 2020-09-12T14:26:48.035 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.036 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4278.0:1", 2020-09-12T14:26:48.036 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.036 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:43.719193+0000", 2020-09-12T14:26:48.036 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.036 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.036 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.036 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.036 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.037 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.037 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.037 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.037 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.037 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.037 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.037 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.037 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.038 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.038 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.038 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.038 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.038 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.038 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:f9a0a8a2:::obj-31:head", 2020-09-12T14:26:48.038 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'31", 2020-09-12T14:26:48.039 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.039 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4281.0:1", 2020-09-12T14:26:48.039 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.039 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:43.878194+0000", 2020-09-12T14:26:48.039 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.039 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.039 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.039 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.040 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.040 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.040 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.040 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.040 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.040 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.040 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.040 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.041 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.041 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.041 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.041 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.041 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.041 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.041 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:1bf2562f:::obj-32:head", 2020-09-12T14:26:48.041 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'32", 2020-09-12T14:26:48.042 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.042 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4284.0:1", 2020-09-12T14:26:48.042 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.042 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:43.995312+0000", 2020-09-12T14:26:48.042 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.042 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.042 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.043 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.043 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.043 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.043 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.043 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.043 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.043 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.043 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.044 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.044 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.044 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.044 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.044 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.044 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.044 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.044 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:b982ae22:::obj-33:head", 2020-09-12T14:26:48.045 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'33", 2020-09-12T14:26:48.045 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.045 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4287.0:1", 2020-09-12T14:26:48.045 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.045 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:44.121660+0000", 2020-09-12T14:26:48.045 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.045 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.046 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.046 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.046 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.046 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.046 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.046 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.046 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.046 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.047 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.047 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.047 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.047 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.047 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.047 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.047 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.048 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.048 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:1ad9f98e:::obj-34:head", 2020-09-12T14:26:48.048 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'34", 2020-09-12T14:26:48.048 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.048 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4290.0:1", 2020-09-12T14:26:48.048 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.048 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:44.272182+0000", 2020-09-12T14:26:48.048 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.049 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.049 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.049 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.049 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.049 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.049 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.049 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.049 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.050 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.050 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.050 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.050 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.050 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.050 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.050 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.051 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.051 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.051 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:941f43ff:::obj-35:head", 2020-09-12T14:26:48.051 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'35", 2020-09-12T14:26:48.051 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.051 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4293.0:1", 2020-09-12T14:26:48.051 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.051 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:44.398266+0000", 2020-09-12T14:26:48.052 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.052 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.052 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.052 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.052 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.052 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.052 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.052 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.053 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.053 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.053 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.053 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.053 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.053 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.053 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.053 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.054 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.054 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.054 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:24ebabe9:::obj-36:head", 2020-09-12T14:26:48.054 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'36", 2020-09-12T14:26:48.054 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.054 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4296.0:1", 2020-09-12T14:26:48.054 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.055 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:44.523763+0000", 2020-09-12T14:26:48.055 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.055 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.055 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.055 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.055 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.055 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.055 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.056 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.056 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.056 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.056 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.056 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.056 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.056 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.056 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.057 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.057 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.057 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.057 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:5f3a5398:::obj-37:head", 2020-09-12T14:26:48.057 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'37", 2020-09-12T14:26:48.057 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.057 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4299.0:1", 2020-09-12T14:26:48.057 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.058 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:44.675070+0000", 2020-09-12T14:26:48.058 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.058 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.058 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.058 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.058 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.058 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.059 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.059 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.059 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.059 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.059 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.059 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.059 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.059 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.060 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.060 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.060 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.060 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.060 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:c96cb37e:::obj-38:head", 2020-09-12T14:26:48.060 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'38", 2020-09-12T14:26:48.060 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.060 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4302.0:1", 2020-09-12T14:26:48.061 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.061 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:44.808854+0000", 2020-09-12T14:26:48.061 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.061 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.061 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.061 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.061 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.062 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.062 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.062 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.062 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.062 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.062 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.062 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.062 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.063 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.063 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.063 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.063 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.063 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.063 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:69e36996:::obj-39:head", 2020-09-12T14:26:48.063 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'39", 2020-09-12T14:26:48.064 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.064 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4305.0:1", 2020-09-12T14:26:48.064 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.064 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:44.942727+0000", 2020-09-12T14:26:48.064 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.064 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.064 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.064 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.065 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.065 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.065 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.065 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.065 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.065 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.065 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.065 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.066 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.066 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.066 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.066 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.066 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.066 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.066 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:9f75773e:::obj-40:head", 2020-09-12T14:26:48.067 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'40", 2020-09-12T14:26:48.067 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.067 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4308.0:1", 2020-09-12T14:26:48.067 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.067 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:45.069667+0000", 2020-09-12T14:26:48.067 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.067 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.067 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.068 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.068 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.068 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.068 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.068 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.068 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.068 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.068 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.069 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.069 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.069 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.069 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.069 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.069 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.069 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.069 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:2c01cd48:::obj-41:head", 2020-09-12T14:26:48.070 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'41", 2020-09-12T14:26:48.070 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.070 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4311.0:1", 2020-09-12T14:26:48.070 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.070 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:45.195057+0000", 2020-09-12T14:26:48.070 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.070 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.071 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.071 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.071 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.071 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.071 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.071 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.071 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.071 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.072 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.072 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.072 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.072 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.072 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.072 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.072 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.072 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.073 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:5e676e3d:::obj-42:head", 2020-09-12T14:26:48.073 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'42", 2020-09-12T14:26:48.073 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.073 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4314.0:1", 2020-09-12T14:26:48.073 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.073 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:45.328377+0000", 2020-09-12T14:26:48.073 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.073 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.074 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.074 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.074 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.074 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.074 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.074 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.074 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.075 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.075 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.075 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.075 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.075 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.075 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.075 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.075 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.076 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.076 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:1215f697:::obj-43:head", 2020-09-12T14:26:48.076 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'43", 2020-09-12T14:26:48.076 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.076 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4317.0:1", 2020-09-12T14:26:48.076 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.076 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:45.454886+0000", 2020-09-12T14:26:48.076 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.077 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.077 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.077 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.077 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.077 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.077 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.077 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.078 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.078 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.078 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.078 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.078 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.078 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.078 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.078 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.079 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.079 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.079 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:3a0aa357:::obj-44:head", 2020-09-12T14:26:48.079 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'44", 2020-09-12T14:26:48.079 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.079 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4320.0:1", 2020-09-12T14:26:48.079 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.079 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:45.588384+0000", 2020-09-12T14:26:48.080 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.080 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.080 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.080 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.080 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.080 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.080 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.081 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.081 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.081 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.081 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.081 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.081 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.081 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.081 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.082 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.082 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.082 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.082 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:6366950a:::obj-45:head", 2020-09-12T14:26:48.082 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'45", 2020-09-12T14:26:48.082 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.082 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4323.0:1", 2020-09-12T14:26:48.082 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.083 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:45.721975+0000", 2020-09-12T14:26:48.083 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.083 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.083 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.083 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.083 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.083 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.083 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.084 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.084 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.084 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.084 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.084 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.084 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.084 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.085 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.085 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.085 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.085 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.085 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:621cb2b3:::obj-46:head", 2020-09-12T14:26:48.085 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'46", 2020-09-12T14:26:48.085 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.085 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4326.0:1", 2020-09-12T14:26:48.086 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.086 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:45.847069+0000", 2020-09-12T14:26:48.086 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.086 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.086 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.086 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.086 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.086 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.087 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.087 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.087 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.087 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.087 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.087 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.087 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.087 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.088 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.088 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.088 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.088 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.088 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:d83ab3cd:::obj-47:head", 2020-09-12T14:26:48.088 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'47", 2020-09-12T14:26:48.088 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.089 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4329.0:1", 2020-09-12T14:26:48.089 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.089 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:45.983070+0000", 2020-09-12T14:26:48.089 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.089 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.089 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.089 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.089 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.090 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.090 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.090 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.090 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.090 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.090 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.090 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.091 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.091 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.091 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.091 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.091 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.091 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.091 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:3d8e6d4a:::obj-48:head", 2020-09-12T14:26:48.091 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'48", 2020-09-12T14:26:48.092 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.092 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4332.0:1", 2020-09-12T14:26:48.092 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.092 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:46.108874+0000", 2020-09-12T14:26:48.092 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.092 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.092 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.092 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.093 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.093 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.093 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.093 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.093 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.093 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.093 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.094 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.094 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.094 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.094 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.094 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.094 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.094 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.094 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:50b1e2f2:::obj-49:head", 2020-09-12T14:26:48.095 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'49", 2020-09-12T14:26:48.095 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.095 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4335.0:1", 2020-09-12T14:26:48.095 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.095 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:46.259819+0000", 2020-09-12T14:26:48.095 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.095 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.095 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.096 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.096 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.096 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.096 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.096 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.096 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.096 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.096 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.097 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.097 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.097 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.097 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.097 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.097 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.097 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.098 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:c75430e6:::obj-50:head", 2020-09-12T14:26:48.098 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'50", 2020-09-12T14:26:48.098 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.098 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4338.0:1", 2020-09-12T14:26:48.098 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.098 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:46.384816+0000", 2020-09-12T14:26:48.098 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.098 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.099 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.099 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.099 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.099 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.099 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.099 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.099 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.099 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.100 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.100 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.100 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.100 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.100 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.100 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.100 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.101 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.101 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:eaf3ca0a:::obj-51:head", 2020-09-12T14:26:48.101 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'51", 2020-09-12T14:26:48.101 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.101 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4341.0:1", 2020-09-12T14:26:48.101 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.101 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:46.510764+0000", 2020-09-12T14:26:48.101 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.102 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.102 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.102 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.102 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.102 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.102 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.102 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.102 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.103 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.103 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.103 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.103 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.103 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.103 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.103 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.103 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.104 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.104 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:86a15bea:::obj-52:head", 2020-09-12T14:26:48.104 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'52", 2020-09-12T14:26:48.104 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.104 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4344.0:1", 2020-09-12T14:26:48.104 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.104 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:46.628542+0000", 2020-09-12T14:26:48.105 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.105 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.105 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.105 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.105 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.105 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.105 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.105 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.106 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.106 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.106 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.106 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.106 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.106 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.106 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.107 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.107 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.107 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.107 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:1df15dae:::obj-53:head", 2020-09-12T14:26:48.107 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'53", 2020-09-12T14:26:48.107 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.107 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4347.0:1", 2020-09-12T14:26:48.107 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.108 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:46.746092+0000", 2020-09-12T14:26:48.108 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.108 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.108 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.108 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.108 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.108 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.108 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.109 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.109 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.109 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.109 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.109 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.109 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.109 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.110 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.110 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.110 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.110 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.110 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:29304dc8:::obj-54:head", 2020-09-12T14:26:48.110 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'54", 2020-09-12T14:26:48.110 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.110 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4350.0:1", 2020-09-12T14:26:48.111 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.111 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:46.880496+0000", 2020-09-12T14:26:48.111 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.111 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.111 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.111 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.111 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.111 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.112 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.112 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.112 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.112 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.112 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.112 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.112 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.113 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.113 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.113 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.113 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.113 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.113 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:de02c26d:::obj-55:head", 2020-09-12T14:26:48.113 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'55", 2020-09-12T14:26:48.113 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.114 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4353.0:1", 2020-09-12T14:26:48.114 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.114 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:47.013802+0000", 2020-09-12T14:26:48.114 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.114 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.114 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.114 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.114 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.115 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.115 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.115 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.115 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.115 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.115 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.115 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.115 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.116 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.116 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.116 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.116 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.116 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.116 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:0d36fc08:::obj-56:head", 2020-09-12T14:26:48.116 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'56", 2020-09-12T14:26:48.116 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.117 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4356.0:1", 2020-09-12T14:26:48.117 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.117 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:47.140811+0000", 2020-09-12T14:26:48.117 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.117 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.117 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.117 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.118 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.118 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.118 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.118 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.118 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.118 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.118 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.118 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.119 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.119 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.119 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.119 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.119 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.119 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.119 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:2b2cbcba:::obj-57:head", 2020-09-12T14:26:48.120 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'57", 2020-09-12T14:26:48.120 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.120 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4359.0:1", 2020-09-12T14:26:48.120 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.120 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:47.281985+0000", 2020-09-12T14:26:48.120 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.120 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.120 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.121 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.121 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.121 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.121 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.121 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.121 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.121 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.121 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.122 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.122 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.122 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.122 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.122 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.122 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.122 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.123 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:838be76e:::obj-58:head", 2020-09-12T14:26:48.123 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'58", 2020-09-12T14:26:48.123 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.123 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4362.0:1", 2020-09-12T14:26:48.123 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.123 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:47.424767+0000", 2020-09-12T14:26:48.123 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.123 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.124 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.124 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.124 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.124 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.124 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.124 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.124 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.124 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.125 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.125 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.125 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.125 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.125 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.125 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.125 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.125 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.126 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:2c66fd6f:::obj-59:head", 2020-09-12T14:26:48.126 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'59", 2020-09-12T14:26:48.126 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.126 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4365.0:1", 2020-09-12T14:26:48.126 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.126 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:47.559574+0000", 2020-09-12T14:26:48.126 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.127 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.127 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.127 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.127 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.127 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.127 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.127 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.127 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.128 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.128 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.128 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.128 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.128 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.128 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.128 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.128 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.129 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.129 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:2b18273b:::obj-60:head", 2020-09-12T14:26:48.129 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'60", 2020-09-12T14:26:48.129 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.129 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4368.0:1", 2020-09-12T14:26:48.129 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.129 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:47.826159+0000", 2020-09-12T14:26:48.129 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.130 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.130 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.130 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.130 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.130 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.130 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.130 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.131 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.131 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.131 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.131 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.131 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.131 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.131 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.131 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.132 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.132 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.132 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:fcfa3cd1:::obj-61:head", 2020-09-12T14:26:48.132 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'61", 2020-09-12T14:26:48.132 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.132 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4371.0:1", 2020-09-12T14:26:48.132 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.132 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:47.953802+0000", 2020-09-12T14:26:48.133 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.133 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.133 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.133 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.133 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.133 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.133 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.133 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.134 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.134 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.134 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.134 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.134 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.134 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.134 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.135 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.135 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.135 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.135 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:16e51a68:::obj-62:head", 2020-09-12T14:26:48.135 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'62", 2020-09-12T14:26:48.135 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.135 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4374.0:1", 2020-09-12T14:26:48.135 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.136 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:48.077234+0000", 2020-09-12T14:26:48.136 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.136 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.136 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.136 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.136 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.136 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.136 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.137 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.137 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.137 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.137 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.137 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.137 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.137 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.137 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.138 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.138 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.138 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.138 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:c15678fc:::obj-63:head", 2020-09-12T14:26:48.138 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'63", 2020-09-12T14:26:48.138 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.138 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4377.0:1", 2020-09-12T14:26:48.139 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.139 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:48.195208+0000", 2020-09-12T14:26:48.139 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.139 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.139 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.139 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.139 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.139 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.140 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.140 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.140 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.140 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.140 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.140 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.140 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.140 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.141 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.141 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.141 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.141 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.141 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:e066252d:::obj-64:head", 2020-09-12T14:26:48.141 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'64", 2020-09-12T14:26:48.141 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.141 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4380.0:1", 2020-09-12T14:26:48.142 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.142 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:48.313774+0000", 2020-09-12T14:26:48.142 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.142 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.142 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.142 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.142 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.143 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.143 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.143 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.143 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.143 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.143 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.143 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.143 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.144 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.144 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.144 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.144 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.144 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.144 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:95a4ceaf:::obj-65:head", 2020-09-12T14:26:48.144 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'65", 2020-09-12T14:26:48.144 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.145 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4383.0:1", 2020-09-12T14:26:48.145 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.145 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:48.447493+0000", 2020-09-12T14:26:48.145 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.145 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.145 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.145 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.145 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.146 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.146 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.146 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.146 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.146 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.146 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.146 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.147 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.147 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.147 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.147 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.147 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.147 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.147 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:6e97b31a:::obj-66:head", 2020-09-12T14:26:48.147 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'66", 2020-09-12T14:26:48.148 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.148 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4386.0:1", 2020-09-12T14:26:48.148 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.148 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:48.581293+0000", 2020-09-12T14:26:48.148 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.148 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.148 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.148 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.149 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.149 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.149 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.149 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.149 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.149 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.149 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.149 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.150 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.150 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.150 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.150 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.150 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.150 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.150 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:ceed8b4e:::obj-67:head", 2020-09-12T14:26:48.151 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'67", 2020-09-12T14:26:48.151 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.151 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4389.0:1", 2020-09-12T14:26:48.151 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.151 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:48.707022+0000", 2020-09-12T14:26:48.151 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.151 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.151 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.152 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.152 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.152 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.152 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.152 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.152 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.152 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.152 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.153 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.153 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.153 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.153 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.153 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.153 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.153 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.154 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:05e2e722:::obj-68:head", 2020-09-12T14:26:48.154 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'68", 2020-09-12T14:26:48.154 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.154 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4392.0:1", 2020-09-12T14:26:48.154 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.154 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:48.841454+0000", 2020-09-12T14:26:48.154 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.154 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.155 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.155 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.155 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.155 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.155 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.155 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.155 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.155 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.156 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.156 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.156 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.156 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.156 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.156 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.156 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.156 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.157 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:c40f7de9:::obj-69:head", 2020-09-12T14:26:48.157 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'69", 2020-09-12T14:26:48.157 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.157 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4395.0:1", 2020-09-12T14:26:48.157 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.157 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:48.983835+0000", 2020-09-12T14:26:48.157 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.157 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.158 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.158 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.158 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.158 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.158 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.158 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.158 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.159 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.159 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.159 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.159 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.159 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.159 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.159 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.159 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.160 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.160 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:7bab3581:::obj-70:head", 2020-09-12T14:26:48.160 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'70", 2020-09-12T14:26:48.160 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.160 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4398.0:1", 2020-09-12T14:26:48.160 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.160 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:49.126421+0000", 2020-09-12T14:26:48.160 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.161 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.161 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.161 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.161 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.161 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.161 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.161 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.161 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.162 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.162 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.162 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.162 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.162 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.162 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.162 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.163 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.163 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.163 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:cb9edc4f:::obj-71:head", 2020-09-12T14:26:48.163 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'71", 2020-09-12T14:26:48.163 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.163 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4401.0:1", 2020-09-12T14:26:48.163 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.163 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:49.268571+0000", 2020-09-12T14:26:48.164 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.164 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.164 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.164 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.164 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.164 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.164 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.164 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.165 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.165 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.165 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.165 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.165 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.165 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.165 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.165 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.166 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.166 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.166 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:88ae729f:::obj-72:head", 2020-09-12T14:26:48.166 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'72", 2020-09-12T14:26:48.166 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.166 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4404.0:1", 2020-09-12T14:26:48.166 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.167 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:49.402662+0000", 2020-09-12T14:26:48.167 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.167 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.167 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.167 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.167 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.167 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.167 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.168 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.168 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.168 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.168 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.168 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.168 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.168 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.168 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.169 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.169 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.169 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.169 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:887b5de4:::obj-73:head", 2020-09-12T14:26:48.169 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'73", 2020-09-12T14:26:48.169 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.169 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4407.0:1", 2020-09-12T14:26:48.169 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.170 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:49.528401+0000", 2020-09-12T14:26:48.170 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.170 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.170 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.170 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.170 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.170 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.170 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.171 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.171 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.171 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.171 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.171 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.171 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.171 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.172 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.172 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.172 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.172 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.172 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:996efeaa:::obj-74:head", 2020-09-12T14:26:48.172 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'74", 2020-09-12T14:26:48.172 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.172 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4410.0:1", 2020-09-12T14:26:48.173 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.173 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:49.656813+0000", 2020-09-12T14:26:48.173 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.173 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.173 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.173 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.173 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.173 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.174 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.174 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.174 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.174 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.174 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.174 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.174 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.175 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.175 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.175 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.175 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.175 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.175 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:942222a0:::obj-75:head", 2020-09-12T14:26:48.175 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'75", 2020-09-12T14:26:48.175 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.176 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4413.0:1", 2020-09-12T14:26:48.176 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.176 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:49.772028+0000", 2020-09-12T14:26:48.176 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.176 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.176 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.176 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.176 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.177 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.177 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.177 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.177 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.177 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.177 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.177 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.177 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.178 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.178 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.178 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.178 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.178 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.178 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:35f0499f:::obj-76:head", 2020-09-12T14:26:48.178 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'76", 2020-09-12T14:26:48.178 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.179 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4416.0:1", 2020-09-12T14:26:48.179 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.179 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:49.906009+0000", 2020-09-12T14:26:48.179 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.179 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.179 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.179 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.179 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.180 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.180 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.180 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.180 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.180 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.180 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.180 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.181 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.181 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.181 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.181 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.181 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.181 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.181 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:15cd38e3:::obj-77:head", 2020-09-12T14:26:48.181 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'77", 2020-09-12T14:26:48.182 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.182 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4419.0:1", 2020-09-12T14:26:48.182 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.182 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:50.037507+0000", 2020-09-12T14:26:48.182 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.182 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.182 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.182 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.183 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.183 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.183 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.183 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.183 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.183 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.183 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.183 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.184 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.184 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.184 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.184 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.184 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.184 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.184 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:7adeb77b:::obj-78:head", 2020-09-12T14:26:48.185 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'78", 2020-09-12T14:26:48.185 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.185 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4422.0:1", 2020-09-12T14:26:48.185 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.185 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:50.174419+0000", 2020-09-12T14:26:48.185 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.185 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.185 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.186 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.186 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.186 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.186 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.186 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.186 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.186 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.186 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.187 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.187 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.187 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.187 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.187 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.187 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.187 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.187 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:c92176ea:::obj-79:head", 2020-09-12T14:26:48.188 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'79", 2020-09-12T14:26:48.188 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.188 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4425.0:1", 2020-09-12T14:26:48.188 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.188 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:50.309213+0000", 2020-09-12T14:26:48.188 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.188 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.189 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.189 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.189 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.189 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.189 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.189 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.189 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.189 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.190 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.190 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.190 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.190 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.190 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.190 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.190 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.190 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.191 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:14abac33:::obj-80:head", 2020-09-12T14:26:48.191 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'80", 2020-09-12T14:26:48.191 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.191 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4428.0:1", 2020-09-12T14:26:48.191 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.191 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:50.451535+0000", 2020-09-12T14:26:48.191 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.191 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.192 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.192 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.192 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.192 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.192 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.192 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.192 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.193 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.193 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.193 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.193 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.193 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.193 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.193 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.193 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.194 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.194 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:ea1005df:::obj-81:head", 2020-09-12T14:26:48.194 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'81", 2020-09-12T14:26:48.194 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.194 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4431.0:1", 2020-09-12T14:26:48.194 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.194 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:50.576206+0000", 2020-09-12T14:26:48.194 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.195 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.195 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.195 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.195 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.195 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.195 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.195 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.195 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.196 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.196 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.196 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.196 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.196 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.196 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.196 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.197 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.197 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.197 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:df18954d:::obj-82:head", 2020-09-12T14:26:48.197 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'82", 2020-09-12T14:26:48.197 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.197 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4434.0:1", 2020-09-12T14:26:48.197 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.197 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:50.694164+0000", 2020-09-12T14:26:48.198 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.198 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.198 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.198 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.198 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.198 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.198 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.198 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.199 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.199 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.199 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.199 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.199 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.199 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.199 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.200 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.200 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.200 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.200 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:cdd3834b:::obj-83:head", 2020-09-12T14:26:48.200 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'83", 2020-09-12T14:26:48.200 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.200 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4437.0:1", 2020-09-12T14:26:48.200 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.201 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:50.818726+0000", 2020-09-12T14:26:48.201 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.201 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.201 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.201 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.201 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.201 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.201 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.202 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.202 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.202 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.202 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.202 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.202 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.202 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.203 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.203 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.203 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.203 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.203 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:dfbf520a:::obj-84:head", 2020-09-12T14:26:48.203 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'84", 2020-09-12T14:26:48.203 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.203 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4440.0:1", 2020-09-12T14:26:48.204 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.204 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:50.936374+0000", 2020-09-12T14:26:48.204 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.204 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.204 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.204 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.204 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.204 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.205 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.205 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.205 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.205 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.205 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.205 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.205 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.205 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.206 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.206 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.206 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.206 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.206 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:1dd3df2f:::obj-85:head", 2020-09-12T14:26:48.206 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'85", 2020-09-12T14:26:48.206 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.207 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4443.0:1", 2020-09-12T14:26:48.207 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.207 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:51.053711+0000", 2020-09-12T14:26:48.207 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.207 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.207 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.207 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.207 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.208 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.208 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.208 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.208 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.208 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.208 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.208 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.208 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.209 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.209 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.209 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.209 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.209 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.209 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:54554b77:::obj-86:head", 2020-09-12T14:26:48.209 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'86", 2020-09-12T14:26:48.209 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.210 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4446.0:1", 2020-09-12T14:26:48.210 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.210 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:51.171003+0000", 2020-09-12T14:26:48.210 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.210 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.210 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.210 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.211 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.211 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.211 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.211 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.211 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.211 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.211 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.211 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.212 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.212 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.212 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.212 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.212 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.212 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.212 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:43a48f98:::obj-87:head", 2020-09-12T14:26:48.212 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'87", 2020-09-12T14:26:48.213 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.213 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4449.0:1", 2020-09-12T14:26:48.213 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.213 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:51.289017+0000", 2020-09-12T14:26:48.213 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.213 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.213 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.213 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.214 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.214 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.214 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.214 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.214 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.214 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.214 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.215 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.215 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.215 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.215 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.215 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.215 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.215 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.215 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:3f7b3ecc:::obj-88:head", 2020-09-12T14:26:48.216 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'88", 2020-09-12T14:26:48.216 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.216 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4452.0:1", 2020-09-12T14:26:48.216 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.216 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:51.423339+0000", 2020-09-12T14:26:48.216 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.216 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.216 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.217 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.217 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.217 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.217 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.217 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.217 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.217 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.217 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.218 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.218 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.218 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.218 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.218 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.218 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.218 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.219 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:d25ef7d2:::obj-89:head", 2020-09-12T14:26:48.219 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'89", 2020-09-12T14:26:48.219 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.219 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4455.0:1", 2020-09-12T14:26:48.219 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.219 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:51.558117+0000", 2020-09-12T14:26:48.219 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.219 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.220 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.220 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.220 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.220 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.220 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.220 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.220 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.220 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.221 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.221 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.221 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.221 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.221 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.221 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.221 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.222 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.222 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:ea27d688:::obj-90:head", 2020-09-12T14:26:48.222 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'90", 2020-09-12T14:26:48.222 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.222 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4458.0:1", 2020-09-12T14:26:48.222 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.222 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:51.708709+0000", 2020-09-12T14:26:48.222 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.223 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.223 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.223 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.223 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.223 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.223 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.223 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.223 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.224 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.224 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.224 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.224 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.224 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.224 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.224 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.224 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.225 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.225 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:088f9d12:::obj-91:head", 2020-09-12T14:26:48.225 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'91", 2020-09-12T14:26:48.225 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.225 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4461.0:1", 2020-09-12T14:26:48.225 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.225 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:51.842689+0000", 2020-09-12T14:26:48.226 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.226 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.226 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.226 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.226 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.226 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.226 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.226 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.227 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.227 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.227 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.227 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.227 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.227 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.227 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.227 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.228 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.228 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.228 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:206f25a5:::obj-92:head", 2020-09-12T14:26:48.228 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'92", 2020-09-12T14:26:48.228 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.228 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4464.0:1", 2020-09-12T14:26:48.228 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.228 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:51.985960+0000", 2020-09-12T14:26:48.229 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.229 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.229 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.229 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.229 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.229 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.229 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.230 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.230 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.230 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.230 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.230 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.230 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.230 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.230 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.231 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.231 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.231 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.231 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:501cca61:::obj-93:head", 2020-09-12T14:26:48.231 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'93", 2020-09-12T14:26:48.231 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.231 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4467.0:1", 2020-09-12T14:26:48.232 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.232 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:52.111255+0000", 2020-09-12T14:26:48.232 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.232 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.232 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.232 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.232 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.232 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.233 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.233 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.233 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.233 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.233 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.233 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.233 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.233 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.234 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.234 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.234 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.234 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.234 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:1ed468c0:::obj-94:head", 2020-09-12T14:26:48.234 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'94", 2020-09-12T14:26:48.234 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.234 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4470.0:1", 2020-09-12T14:26:48.235 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.235 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:52.245164+0000", 2020-09-12T14:26:48.235 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.235 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.235 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.235 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.235 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.236 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.236 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.236 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.236 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.236 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.236 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.236 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.236 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.237 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.237 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.237 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.237 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.237 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.237 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:e8cba918:::obj-95:head", 2020-09-12T14:26:48.237 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'95", 2020-09-12T14:26:48.237 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.238 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4473.0:1", 2020-09-12T14:26:48.238 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.238 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:52.370255+0000", 2020-09-12T14:26:48.238 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.238 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.238 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.238 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.239 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.239 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.239 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.239 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.239 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.239 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.239 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.239 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.240 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.240 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.240 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.240 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.240 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.240 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.240 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:403c3dfc:::obj-96:head", 2020-09-12T14:26:48.240 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'96", 2020-09-12T14:26:48.241 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.241 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4476.0:1", 2020-09-12T14:26:48.241 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.241 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:52.488559+0000", 2020-09-12T14:26:48.241 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.241 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.241 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.241 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.242 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.242 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.242 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.242 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.242 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.242 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.242 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.243 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.243 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.243 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.243 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.243 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.243 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.243 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.243 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:7c2d0cb1:::obj-97:head", 2020-09-12T14:26:48.244 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'97", 2020-09-12T14:26:48.244 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.244 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4479.0:1", 2020-09-12T14:26:48.244 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.244 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:52.605304+0000", 2020-09-12T14:26:48.244 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.244 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.244 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.245 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.245 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.245 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.245 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.245 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.245 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.245 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.246 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.246 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.246 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.246 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.246 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.246 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.246 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.246 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.247 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:6f3e26a7:::obj-98:head", 2020-09-12T14:26:48.247 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'98", 2020-09-12T14:26:48.247 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.247 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4482.0:1", 2020-09-12T14:26:48.247 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.247 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:52.721847+0000", 2020-09-12T14:26:48.247 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.247 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.248 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.248 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.248 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.248 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.248 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.248 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.248 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.249 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.249 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.249 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.249 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.249 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.249 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.249 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.249 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.250 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.250 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:a4b22c6a:::obj-99:head", 2020-09-12T14:26:48.250 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'99", 2020-09-12T14:26:48.250 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.250 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4485.0:1", 2020-09-12T14:26:48.250 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.250 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:52.855768+0000", 2020-09-12T14:26:48.250 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.251 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.251 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.251 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.251 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.251 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.251 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.251 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.251 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.252 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.252 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.252 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.252 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.252 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.252 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.252 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.253 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.253 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.253 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:25c09ea8:::obj-100:head", 2020-09-12T14:26:48.253 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'100", 2020-09-12T14:26:48.253 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.253 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4488.0:1", 2020-09-12T14:26:48.253 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.253 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:52.989778+0000", 2020-09-12T14:26:48.254 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.254 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.254 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.254 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.254 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.254 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.254 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.254 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.255 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.255 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.255 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.255 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.255 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.255 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.255 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.256 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.256 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.256 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.256 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:be8b0193:::obj-101:head", 2020-09-12T14:26:48.256 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'101", 2020-09-12T14:26:48.256 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.256 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4491.0:1", 2020-09-12T14:26:48.256 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.257 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:53.140995+0000", 2020-09-12T14:26:48.257 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.257 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.257 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.257 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.257 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.257 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.257 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.258 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.258 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.258 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.258 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.258 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.258 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.258 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.259 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.259 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.259 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.259 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.259 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:6aa5e219:::obj-102:head", 2020-09-12T14:26:48.259 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'102", 2020-09-12T14:26:48.259 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.259 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4494.0:1", 2020-09-12T14:26:48.260 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.260 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:53.276435+0000", 2020-09-12T14:26:48.260 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.260 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.260 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.260 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.260 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.260 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.261 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.261 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.261 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.261 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.261 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.261 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.261 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.261 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.262 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.262 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.262 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.262 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.262 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:ff8cd32f:::obj-103:head", 2020-09-12T14:26:48.262 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'103", 2020-09-12T14:26:48.262 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.263 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4497.0:1", 2020-09-12T14:26:48.263 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.263 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:53.402013+0000", 2020-09-12T14:26:48.263 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.263 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.263 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.263 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.263 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.264 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.264 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.264 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.264 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.264 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.264 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.264 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.264 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.265 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.265 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.265 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.265 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.265 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.265 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:fcaf5c1c:::obj-104:head", 2020-09-12T14:26:48.265 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'104", 2020-09-12T14:26:48.266 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.266 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4500.0:1", 2020-09-12T14:26:48.266 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.266 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:53.519948+0000", 2020-09-12T14:26:48.266 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.266 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.266 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.266 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.267 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.267 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.267 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.267 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.267 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.267 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.267 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.267 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.268 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.268 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.268 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.268 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.268 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.268 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.268 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:2111aa60:::obj-105:head", 2020-09-12T14:26:48.268 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'105", 2020-09-12T14:26:48.269 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.269 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4503.0:1", 2020-09-12T14:26:48.269 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.269 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:53.637530+0000", 2020-09-12T14:26:48.269 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.269 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.269 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.270 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.270 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.270 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.270 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.270 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.270 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.270 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.270 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.271 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.271 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.271 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.271 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.271 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.271 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.271 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.271 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:9fed4eb7:::obj-106:head", 2020-09-12T14:26:48.272 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'106", 2020-09-12T14:26:48.272 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.272 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4506.0:1", 2020-09-12T14:26:48.272 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.272 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:53.761125+0000", 2020-09-12T14:26:48.272 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.272 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.272 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.273 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.273 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.273 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.273 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.273 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.273 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.273 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.274 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.274 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.274 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.274 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.274 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.274 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.274 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.274 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.275 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:0b0c0bce:::obj-107:head", 2020-09-12T14:26:48.275 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'107", 2020-09-12T14:26:48.275 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.275 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4509.0:1", 2020-09-12T14:26:48.275 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.275 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:53.889087+0000", 2020-09-12T14:26:48.275 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.275 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.276 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.276 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.276 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.276 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.276 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.276 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.276 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.277 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.277 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.277 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.277 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.277 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.277 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.277 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.277 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.278 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.278 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:b7ce613a:::obj-108:head", 2020-09-12T14:26:48.278 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'108", 2020-09-12T14:26:48.278 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.278 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4512.0:1", 2020-09-12T14:26:48.278 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.278 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:54.004843+0000", 2020-09-12T14:26:48.278 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.279 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.279 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.279 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.279 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.279 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.279 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.279 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.279 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.280 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.280 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.280 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.280 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.280 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.280 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.280 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.281 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.281 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.281 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:4f32cd61:::obj-109:head", 2020-09-12T14:26:48.281 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'109", 2020-09-12T14:26:48.281 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.281 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4515.0:1", 2020-09-12T14:26:48.281 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.281 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:54.121733+0000", 2020-09-12T14:26:48.282 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.282 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.282 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.282 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.282 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.282 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.282 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.282 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.283 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.283 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.283 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.283 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.283 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.283 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.283 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.283 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.284 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.284 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.284 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:40f97f88:::obj-110:head", 2020-09-12T14:26:48.284 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'110", 2020-09-12T14:26:48.284 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.284 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4518.0:1", 2020-09-12T14:26:48.284 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.284 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:54.264563+0000", 2020-09-12T14:26:48.285 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.285 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.285 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.285 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.285 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.285 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.285 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.286 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.286 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.286 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.286 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.286 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.286 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.286 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.286 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.287 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.287 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.287 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.287 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:1d8e7a62:::obj-111:head", 2020-09-12T14:26:48.287 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'111", 2020-09-12T14:26:48.287 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.287 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4521.0:1", 2020-09-12T14:26:48.288 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.288 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:54.407011+0000", 2020-09-12T14:26:48.288 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.288 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.288 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.288 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.288 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.288 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.289 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.289 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.289 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.289 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.289 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.289 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.289 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.289 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.290 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.290 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.290 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.290 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.290 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:3a283967:::obj-112:head", 2020-09-12T14:26:48.290 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'112", 2020-09-12T14:26:48.290 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.290 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4524.0:1", 2020-09-12T14:26:48.291 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.291 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:54.532895+0000", 2020-09-12T14:26:48.291 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.291 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.291 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.291 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.291 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.292 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.292 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.292 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.292 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.292 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.292 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.292 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.292 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.293 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.293 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.293 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.293 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.293 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.293 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:2acdc9e7:::obj-113:head", 2020-09-12T14:26:48.293 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'113", 2020-09-12T14:26:48.293 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.294 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4527.0:1", 2020-09-12T14:26:48.294 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.294 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:54.650740+0000", 2020-09-12T14:26:48.294 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.294 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.294 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.294 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.294 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.295 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.295 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.295 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.295 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.295 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.295 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.295 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.296 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.296 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.296 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.296 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.296 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.296 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.296 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:7f18a9ad:::obj-114:head", 2020-09-12T14:26:48.296 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'114", 2020-09-12T14:26:48.297 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.297 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4530.0:1", 2020-09-12T14:26:48.297 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.297 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:54.793758+0000", 2020-09-12T14:26:48.297 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.297 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.297 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.297 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.298 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.298 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.298 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.298 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.298 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.298 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.298 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.298 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.299 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.299 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.299 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.299 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.299 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.299 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.299 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:852b8a34:::obj-115:head", 2020-09-12T14:26:48.300 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'115", 2020-09-12T14:26:48.300 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.300 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4533.0:1", 2020-09-12T14:26:48.300 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.300 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:54.919628+0000", 2020-09-12T14:26:48.300 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.300 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.300 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.301 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.301 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.301 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.301 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.301 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.301 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.301 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.301 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.302 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.302 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.302 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.302 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.302 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.302 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.302 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.303 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:589de4f7:::obj-116:head", 2020-09-12T14:26:48.303 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'116", 2020-09-12T14:26:48.303 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.303 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4536.0:1", 2020-09-12T14:26:48.303 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.303 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:55.037126+0000", 2020-09-12T14:26:48.303 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.303 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.304 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.304 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.304 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.304 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.304 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.304 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.304 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.304 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.305 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.305 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.305 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.305 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.305 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.305 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.305 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.305 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.306 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:14f2571c:::obj-117:head", 2020-09-12T14:26:48.306 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'117", 2020-09-12T14:26:48.306 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.306 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4539.0:1", 2020-09-12T14:26:48.306 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.306 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:55.172331+0000", 2020-09-12T14:26:48.306 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.307 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.307 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.307 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.307 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.307 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.307 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.307 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.307 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.308 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.308 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.308 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.308 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.308 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.308 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.308 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.308 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.309 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.309 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:dd36f0e5:::obj-118:head", 2020-09-12T14:26:48.309 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'118", 2020-09-12T14:26:48.309 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.309 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4542.0:1", 2020-09-12T14:26:48.309 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.309 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:55.301858+0000", 2020-09-12T14:26:48.309 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.310 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.310 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.310 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.310 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.310 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.310 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.310 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.311 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.311 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.311 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.311 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.311 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.311 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.311 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.311 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.312 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.312 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.312 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:0d7fa432:::obj-119:head", 2020-09-12T14:26:48.312 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'119", 2020-09-12T14:26:48.312 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.312 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4545.0:1", 2020-09-12T14:26:48.312 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.312 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:55.432113+0000", 2020-09-12T14:26:48.313 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.313 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.313 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.313 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.313 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.313 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.313 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.313 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.314 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.314 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.314 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.314 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.314 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.314 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.314 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.315 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.315 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.315 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.315 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:295ec31a:::obj-120:head", 2020-09-12T14:26:48.315 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'120", 2020-09-12T14:26:48.315 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.315 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4548.0:1", 2020-09-12T14:26:48.315 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.316 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:55.558624+0000", 2020-09-12T14:26:48.316 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.316 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.316 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.316 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.316 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.316 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.316 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.317 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.317 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.317 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.317 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.317 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.317 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.317 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.317 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.318 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.318 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.318 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.318 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:c2863561:::obj-121:head", 2020-09-12T14:26:48.318 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'121", 2020-09-12T14:26:48.318 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.318 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4551.0:1", 2020-09-12T14:26:48.319 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.319 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:55.701186+0000", 2020-09-12T14:26:48.319 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.319 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.319 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.319 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.319 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.319 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.320 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.320 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.320 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.320 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.320 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.320 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.320 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.320 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.321 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.321 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.321 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.321 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.321 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:0b943493:::obj-122:head", 2020-09-12T14:26:48.321 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'122", 2020-09-12T14:26:48.321 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.322 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4554.0:1", 2020-09-12T14:26:48.322 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.322 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:55.843134+0000", 2020-09-12T14:26:48.322 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.322 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.322 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.322 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.322 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.323 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.323 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.323 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.323 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.323 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.323 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.323 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.323 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.324 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.324 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.324 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.324 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.324 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.324 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:4cfecbf8:::obj-123:head", 2020-09-12T14:26:48.324 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'123", 2020-09-12T14:26:48.325 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.325 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4557.0:1", 2020-09-12T14:26:48.325 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.325 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:55.978015+0000", 2020-09-12T14:26:48.325 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.325 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.325 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.325 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.326 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.326 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.326 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.326 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.326 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.326 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.326 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.326 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.327 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.327 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.327 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.327 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.327 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.327 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.327 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:fd18e511:::obj-124:head", 2020-09-12T14:26:48.327 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'124", 2020-09-12T14:26:48.328 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.328 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4560.0:1", 2020-09-12T14:26:48.328 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.328 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:56.128353+0000", 2020-09-12T14:26:48.328 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.328 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.328 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.329 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.329 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.329 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.329 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.329 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.329 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.329 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.329 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.330 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.330 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.330 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.330 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.330 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.330 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.330 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.330 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:4f299322:::obj-125:head", 2020-09-12T14:26:48.331 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'125", 2020-09-12T14:26:48.331 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.331 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4563.0:1", 2020-09-12T14:26:48.331 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.331 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:56.255268+0000", 2020-09-12T14:26:48.331 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.331 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.332 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.332 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.332 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.332 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.332 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.332 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.332 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.332 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.333 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.333 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.333 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.333 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.333 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.333 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.333 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.333 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.334 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:302ecc7e:::obj-126:head", 2020-09-12T14:26:48.334 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'126", 2020-09-12T14:26:48.334 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.334 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4566.0:1", 2020-09-12T14:26:48.334 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.334 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:56.387264+0000", 2020-09-12T14:26:48.334 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.334 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.335 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.335 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.335 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.335 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.335 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.335 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.335 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.336 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.336 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.336 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.336 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.336 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.336 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.336 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.336 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.337 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.337 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:42feb657:::obj-127:head", 2020-09-12T14:26:48.337 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'127", 2020-09-12T14:26:48.337 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.337 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4569.0:1", 2020-09-12T14:26:48.337 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.337 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:56.513976+0000", 2020-09-12T14:26:48.337 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.338 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.338 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.338 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.338 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.338 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.338 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.338 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.338 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.339 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.339 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.339 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.339 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.339 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.339 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.339 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.340 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.340 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.340 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:5e6b9a81:::obj-128:head", 2020-09-12T14:26:48.340 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'128", 2020-09-12T14:26:48.340 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.340 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4572.0:1", 2020-09-12T14:26:48.340 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.340 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:56.639656+0000", 2020-09-12T14:26:48.341 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.341 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.341 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.341 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.341 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.341 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.341 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.341 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.342 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.342 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.342 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.342 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.342 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.342 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.342 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.343 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.343 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.343 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.343 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:0d242242:::obj-129:head", 2020-09-12T14:26:48.343 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'129", 2020-09-12T14:26:48.343 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.343 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4575.0:1", 2020-09-12T14:26:48.343 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.344 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:56.765964+0000", 2020-09-12T14:26:48.344 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.344 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.344 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.344 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.344 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.344 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.344 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.345 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.345 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.345 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.345 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.345 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.345 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.345 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.345 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.346 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.346 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.346 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.346 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:ed409294:::obj-130:head", 2020-09-12T14:26:48.346 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'130", 2020-09-12T14:26:48.346 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.346 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4578.0:1", 2020-09-12T14:26:48.347 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.347 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:56.891740+0000", 2020-09-12T14:26:48.347 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.347 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.347 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.347 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.347 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.347 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.348 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.348 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.348 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.348 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.348 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.348 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.348 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.348 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.349 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.349 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.349 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.349 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.349 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:720761ff:::obj-131:head", 2020-09-12T14:26:48.349 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'131", 2020-09-12T14:26:48.349 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.350 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4581.0:1", 2020-09-12T14:26:48.350 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.350 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:57.019399+0000", 2020-09-12T14:26:48.350 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.350 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.350 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.350 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.350 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.351 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.351 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.351 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.351 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.351 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.351 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.351 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.351 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.352 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.352 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.352 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.352 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.352 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.352 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:7715ee72:::obj-132:head", 2020-09-12T14:26:48.352 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'132", 2020-09-12T14:26:48.352 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.353 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4584.0:1", 2020-09-12T14:26:48.353 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.353 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:57.153462+0000", 2020-09-12T14:26:48.353 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.353 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.353 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.353 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.354 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.354 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.354 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.354 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.354 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.354 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.354 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.354 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.355 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.355 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.355 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.355 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.355 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.355 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.355 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:580938e3:::obj-133:head", 2020-09-12T14:26:48.355 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'133", 2020-09-12T14:26:48.356 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.356 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4587.0:1", 2020-09-12T14:26:48.356 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.356 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:57.279270+0000", 2020-09-12T14:26:48.356 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.356 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.356 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.356 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.357 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.357 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.357 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.357 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.357 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.357 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.357 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.358 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.358 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.358 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.358 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.358 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.358 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.358 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.358 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:05dc98b5:::obj-134:head", 2020-09-12T14:26:48.359 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'134", 2020-09-12T14:26:48.359 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.359 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4590.0:1", 2020-09-12T14:26:48.359 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.359 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:57.421434+0000", 2020-09-12T14:26:48.359 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.359 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.359 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.360 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.360 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.360 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.360 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.360 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.360 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.360 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.361 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.361 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.361 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.361 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.361 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.361 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.361 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.361 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.362 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:73221e51:::obj-135:head", 2020-09-12T14:26:48.362 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'135", 2020-09-12T14:26:48.362 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.362 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4593.0:1", 2020-09-12T14:26:48.362 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.362 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:57.547413+0000", 2020-09-12T14:26:48.362 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.363 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.363 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.363 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.363 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.363 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.363 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.363 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.363 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.364 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.364 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.364 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.364 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.364 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.364 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.364 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.364 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.365 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.365 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:3a97ede2:::obj-136:head", 2020-09-12T14:26:48.365 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'136", 2020-09-12T14:26:48.365 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.365 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4596.0:1", 2020-09-12T14:26:48.365 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.365 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:57.682860+0000", 2020-09-12T14:26:48.365 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.366 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.366 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.366 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.366 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.366 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.366 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.366 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.367 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.367 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.367 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.367 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.367 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.367 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.367 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.367 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.368 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.368 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.368 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:9e95a205:::obj-137:head", 2020-09-12T14:26:48.368 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'137", 2020-09-12T14:26:48.368 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.368 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4599.0:1", 2020-09-12T14:26:48.368 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.368 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:57.823958+0000", 2020-09-12T14:26:48.369 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.369 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.369 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.369 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.369 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.369 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.369 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.369 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.370 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.370 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.370 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.370 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.370 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.370 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.370 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.371 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.371 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.371 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.371 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:27f846bd:::obj-138:head", 2020-09-12T14:26:48.371 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'138", 2020-09-12T14:26:48.371 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.371 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4602.0:1", 2020-09-12T14:26:48.371 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.372 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:58.024746+0000", 2020-09-12T14:26:48.372 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.372 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.372 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.372 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.372 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.372 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.372 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.373 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.373 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.373 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.373 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.373 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.373 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.373 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.374 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.374 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.374 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.374 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.374 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:bb2f7605:::obj-139:head", 2020-09-12T14:26:48.374 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'139", 2020-09-12T14:26:48.374 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.374 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4605.0:1", 2020-09-12T14:26:48.375 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.375 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:58.142639+0000", 2020-09-12T14:26:48.375 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.375 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.375 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.375 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.375 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.375 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.376 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.376 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.376 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.376 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.376 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.376 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.376 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.376 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.377 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.377 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.377 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.377 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.377 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:27668d5f:::obj-140:head", 2020-09-12T14:26:48.377 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'140", 2020-09-12T14:26:48.377 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.378 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4608.0:1", 2020-09-12T14:26:48.378 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.378 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:58.276728+0000", 2020-09-12T14:26:48.378 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.378 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.378 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.378 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.378 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.379 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.379 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.379 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.379 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.379 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.379 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.379 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.379 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.380 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.380 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.380 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.380 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.380 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.380 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:0d70cb86:::obj-141:head", 2020-09-12T14:26:48.380 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'141", 2020-09-12T14:26:48.380 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.381 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4611.0:1", 2020-09-12T14:26:48.381 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.381 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:58.411928+0000", 2020-09-12T14:26:48.381 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.381 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.381 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.381 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.381 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.382 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.382 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.382 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.382 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.382 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.382 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.382 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.383 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.383 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.383 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.383 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.383 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.383 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.383 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:347b39b5:::obj-more-1:head", 2020-09-12T14:26:48.383 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "38'142", 2020-09-12T14:26:48.384 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.384 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4618.0:1", 2020-09-12T14:26:48.384 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.384 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:25:59.487882+0000", 2020-09-12T14:26:48.384 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.384 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.384 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.384 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.385 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.385 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.385 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.385 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.385 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.385 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.385 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.385 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.386 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.386 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.386 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.386 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.386 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.386 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.386 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:0c63cc63:::obj-more-2:head", 2020-09-12T14:26:48.387 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "38'143", 2020-09-12T14:26:48.387 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.387 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4621.0:1", 2020-09-12T14:26:48.387 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.387 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:26:03.559142+0000", 2020-09-12T14:26:48.387 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.387 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.387 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.388 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.388 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.388 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.388 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.388 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.388 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.388 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.388 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.389 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.389 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.389 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.389 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.389 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.389 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.389 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.389 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:3ce69b57:::obj-more-3:head", 2020-09-12T14:26:48.390 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "38'144", 2020-09-12T14:26:48.390 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.390 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4624.0:1", 2020-09-12T14:26:48.390 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.390 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:26:03.794969+0000", 2020-09-12T14:26:48.390 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.390 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.391 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.391 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.391 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.391 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.391 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.391 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.391 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.391 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.392 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.392 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.392 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.392 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.392 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.392 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.392 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.392 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.393 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:50476200:::obj-more-4:head", 2020-09-12T14:26:48.393 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "38'145", 2020-09-12T14:26:48.393 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.393 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4627.0:1", 2020-09-12T14:26:48.393 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.393 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:26:04.075762+0000", 2020-09-12T14:26:48.393 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.393 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.394 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.394 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.394 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.394 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.394 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.394 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.394 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.395 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.395 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.395 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.395 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.395 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.395 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.395 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.395 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.396 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.396 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:7a39cfd8:::obj-more-5:head", 2020-09-12T14:26:48.396 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "38'146", 2020-09-12T14:26:48.396 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.396 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4630.0:1", 2020-09-12T14:26:48.396 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.396 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:26:04.348347+0000", 2020-09-12T14:26:48.396 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.397 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.397 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.397 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.397 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.397 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.397 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.397 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.397 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.398 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.398 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.398 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.398 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.398 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.398 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.398 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.399 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.399 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.399 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:a7159f95:::obj-more-6:head", 2020-09-12T14:26:48.399 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "38'147", 2020-09-12T14:26:48.399 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.399 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4633.0:1", 2020-09-12T14:26:48.399 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.399 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:26:04.592024+0000", 2020-09-12T14:26:48.400 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.400 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.400 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.400 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.400 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.400 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.400 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.400 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.401 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.401 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.401 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.401 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.401 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.401 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.401 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.402 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.402 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.402 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.402 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:37f29f5f:::obj-more-7:head", 2020-09-12T14:26:48.402 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "38'148", 2020-09-12T14:26:48.402 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.402 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4636.0:1", 2020-09-12T14:26:48.402 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.403 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:26:04.836189+0000", 2020-09-12T14:26:48.403 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.403 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.403 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.403 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.403 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.403 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.403 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.404 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.404 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.404 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.404 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.404 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.404 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.404 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.404 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.405 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.405 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.405 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.405 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:4e4836c1:::obj-more-8:head", 2020-09-12T14:26:48.405 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "38'149", 2020-09-12T14:26:48.405 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.405 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4639.0:1", 2020-09-12T14:26:48.406 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.406 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:26:05.094476+0000", 2020-09-12T14:26:48.406 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.406 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.406 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.406 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.406 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.406 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.407 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.407 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.407 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.407 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.407 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.407 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.407 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.407 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.408 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.408 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.408 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.408 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.408 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:d56cad57:::obj-more-9:head", 2020-09-12T14:26:48.408 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "38'150", 2020-09-12T14:26:48.408 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.409 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4642.0:1", 2020-09-12T14:26:48.409 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.409 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:26:05.405387+0000", 2020-09-12T14:26:48.409 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.409 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.409 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.409 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.409 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.410 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.410 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.410 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.410 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.410 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.410 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.410 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.410 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.411 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.411 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.411 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.411 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.411 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.411 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:96ff4b77:::obj-more-10:head", 2020-09-12T14:26:48.411 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "38'151", 2020-09-12T14:26:48.411 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.412 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4645.0:1", 2020-09-12T14:26:48.412 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.412 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:26:05.606314+0000", 2020-09-12T14:26:48.412 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.412 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.412 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.412 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.413 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.413 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.413 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.413 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.413 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.413 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.413 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.413 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.414 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.414 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.414 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.414 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.414 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.414 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.414 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:b8937ca5:::obj-more-11:head", 2020-09-12T14:26:48.414 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "38'152", 2020-09-12T14:26:48.415 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.415 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4648.0:1", 2020-09-12T14:26:48.415 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.415 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:26:05.851730+0000", 2020-09-12T14:26:48.415 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.415 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.415 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.415 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.416 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.416 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.416 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.416 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.416 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.416 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.416 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.416 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.417 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.417 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.417 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.417 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.417 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.417 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.417 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:c949902e:::obj-more-12:head", 2020-09-12T14:26:48.418 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "38'153", 2020-09-12T14:26:48.418 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.418 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4651.0:1", 2020-09-12T14:26:48.418 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.418 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:26:06.111300+0000", 2020-09-12T14:26:48.418 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.418 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.418 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.419 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.419 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.419 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.419 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.419 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.419 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.419 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.419 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.420 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.420 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.420 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.420 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.420 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.420 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.420 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.421 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:4909985f:::obj-more-13:head", 2020-09-12T14:26:48.421 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "38'154", 2020-09-12T14:26:48.421 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.421 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4654.0:1", 2020-09-12T14:26:48.421 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.421 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:26:06.362915+0000", 2020-09-12T14:26:48.421 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.421 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.422 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.422 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.422 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.422 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.422 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.422 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.422 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.422 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.423 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.423 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.423 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.423 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.423 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.423 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.423 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.423 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.424 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:acf906ce:::obj-more-14:head", 2020-09-12T14:26:48.424 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "38'155", 2020-09-12T14:26:48.424 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.424 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4657.0:1", 2020-09-12T14:26:48.424 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.424 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:26:06.597652+0000", 2020-09-12T14:26:48.424 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.425 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.425 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.425 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.425 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.425 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.425 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.425 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.425 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.426 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.426 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.426 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.426 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.426 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.426 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.426 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.427 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.427 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.427 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:44d31ae5:::obj-more-15:head", 2020-09-12T14:26:48.427 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "38'156", 2020-09-12T14:26:48.427 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.427 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4660.0:1", 2020-09-12T14:26:48.427 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.427 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:26:06.840840+0000", 2020-09-12T14:26:48.428 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.428 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.428 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.428 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.428 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.428 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.428 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.428 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.429 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.429 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.429 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.429 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.429 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.429 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.429 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.429 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.430 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.430 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.430 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:54dfd3e0:::obj-more-16:head", 2020-09-12T14:26:48.430 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "38'157", 2020-09-12T14:26:48.430 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.430 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4663.0:1", 2020-09-12T14:26:48.430 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.431 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:26:07.091777+0000", 2020-09-12T14:26:48.431 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.431 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.431 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.431 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.431 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.431 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.431 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.432 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.432 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.432 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.432 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.432 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.432 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.432 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.432 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.433 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.433 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.433 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.433 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:6a7dcd50:::obj-more-17:head", 2020-09-12T14:26:48.433 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "38'158", 2020-09-12T14:26:48.433 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.433 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4666.0:1", 2020-09-12T14:26:48.433 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.434 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:26:07.335606+0000", 2020-09-12T14:26:48.434 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.434 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.434 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.434 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.434 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.434 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.435 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.435 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.435 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.435 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.435 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.435 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.435 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.435 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.436 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.436 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.436 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.436 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.436 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:fb20876f:::obj-more-18:head", 2020-09-12T14:26:48.436 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "38'159", 2020-09-12T14:26:48.436 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.436 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4669.0:1", 2020-09-12T14:26:48.437 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.437 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:26:07.579278+0000", 2020-09-12T14:26:48.437 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.437 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.437 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.437 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.437 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.437 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.438 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.438 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.438 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.438 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.438 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.438 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.438 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.439 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.439 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.439 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.439 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.439 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.439 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:ed63ae36:::obj-more-19:head", 2020-09-12T14:26:48.439 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "38'160", 2020-09-12T14:26:48.439 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.440 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4672.0:1", 2020-09-12T14:26:48.440 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.440 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:26:07.823429+0000", 2020-09-12T14:26:48.445 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.445 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.445 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.445 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.445 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.446 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.446 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.446 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.446 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.446 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.446 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.446 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.447 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.447 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.447 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.447 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.447 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.447 INFO:tasks.workunit.client.0.smithi027.stdout: "op": "modify", 2020-09-12T14:26:48.447 INFO:tasks.workunit.client.0.smithi027.stdout: "object": "1:6a8a392a:::obj-more-20:head", 2020-09-12T14:26:48.448 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "38'161", 2020-09-12T14:26:48.448 INFO:tasks.workunit.client.0.smithi027.stdout: "prior_version": "0'0", 2020-09-12T14:26:48.448 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4675.0:1", 2020-09-12T14:26:48.448 INFO:tasks.workunit.client.0.smithi027.stdout: "extra_reqids": [], 2020-09-12T14:26:48.448 INFO:tasks.workunit.client.0.smithi027.stdout: "mtime": "2020-09-12T14:26:08.082681+0000", 2020-09-12T14:26:48.448 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": 0, 2020-09-12T14:26:48.448 INFO:tasks.workunit.client.0.smithi027.stdout: "mod_desc": { 2020-09-12T14:26:48.448 INFO:tasks.workunit.client.0.smithi027.stdout: "object_mod_desc": { 2020-09-12T14:26:48.449 INFO:tasks.workunit.client.0.smithi027.stdout: "can_local_rollback": false, 2020-09-12T14:26:48.449 INFO:tasks.workunit.client.0.smithi027.stdout: "rollback_info_completed": false, 2020-09-12T14:26:48.449 INFO:tasks.workunit.client.0.smithi027.stdout: "ops": [] 2020-09-12T14:26:48.449 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.449 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.449 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_regions": { 2020-09-12T14:26:48.449 INFO:tasks.workunit.client.0.smithi027.stdout: "object_clean_regions": { 2020-09-12T14:26:48.449 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_offsets": "[7730~18446744073709543885]", 2020-09-12T14:26:48.450 INFO:tasks.workunit.client.0.smithi027.stdout: "clean_omap": true, 2020-09-12T14:26:48.450 INFO:tasks.workunit.client.0.smithi027.stdout: "new_object": false 2020-09-12T14:26:48.450 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.450 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.450 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.450 INFO:tasks.workunit.client.0.smithi027.stdout: ], 2020-09-12T14:26:48.450 INFO:tasks.workunit.client.0.smithi027.stdout: "dups": [ 2020-09-12T14:26:48.450 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.451 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4191.0:1", 2020-09-12T14:26:48.451 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "33'1", 2020-09-12T14:26:48.451 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "1", 2020-09-12T14:26:48.451 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:26:48.451 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.451 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.451 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4194.0:1", 2020-09-12T14:26:48.452 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'2", 2020-09-12T14:26:48.452 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "2", 2020-09-12T14:26:48.452 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:26:48.452 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.452 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.452 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4197.0:1", 2020-09-12T14:26:48.452 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'3", 2020-09-12T14:26:48.452 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "3", 2020-09-12T14:26:48.453 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:26:48.453 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.453 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.453 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4200.0:1", 2020-09-12T14:26:48.453 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'4", 2020-09-12T14:26:48.453 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "4", 2020-09-12T14:26:48.453 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:26:48.453 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.454 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.454 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4203.0:1", 2020-09-12T14:26:48.454 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'5", 2020-09-12T14:26:48.454 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "5", 2020-09-12T14:26:48.454 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:26:48.454 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.454 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.454 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4206.0:1", 2020-09-12T14:26:48.455 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'6", 2020-09-12T14:26:48.455 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "6", 2020-09-12T14:26:48.455 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:26:48.455 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.455 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.455 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4209.0:1", 2020-09-12T14:26:48.455 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'7", 2020-09-12T14:26:48.455 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "7", 2020-09-12T14:26:48.456 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:26:48.456 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.456 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.456 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4212.0:1", 2020-09-12T14:26:48.456 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'8", 2020-09-12T14:26:48.456 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "8", 2020-09-12T14:26:48.456 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:26:48.457 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.457 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.457 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4215.0:1", 2020-09-12T14:26:48.457 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'9", 2020-09-12T14:26:48.457 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "9", 2020-09-12T14:26:48.457 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:26:48.457 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.457 INFO:tasks.workunit.client.0.smithi027.stdout: { 2020-09-12T14:26:48.458 INFO:tasks.workunit.client.0.smithi027.stdout: "reqid": "client.4218.0:1", 2020-09-12T14:26:48.458 INFO:tasks.workunit.client.0.smithi027.stdout: "version": "34'10", 2020-09-12T14:26:48.458 INFO:tasks.workunit.client.0.smithi027.stdout: "user_version": "10", 2020-09-12T14:26:48.458 INFO:tasks.workunit.client.0.smithi027.stdout: "return_code": "0" 2020-09-12T14:26:48.458 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.458 INFO:tasks.workunit.client.0.smithi027.stdout: ] 2020-09-12T14:26:48.458 INFO:tasks.workunit.client.0.smithi027.stdout: }, 2020-09-12T14:26:48.458 INFO:tasks.workunit.client.0.smithi027.stdout: "pg_missing_t": { 2020-09-12T14:26:48.459 INFO:tasks.workunit.client.0.smithi027.stdout: "missing": [], 2020-09-12T14:26:48.459 INFO:tasks.workunit.client.0.smithi027.stdout: "may_include_deletes": true 2020-09-12T14:26:48.459 INFO:tasks.workunit.client.0.smithi027.stdout: } 2020-09-12T14:26:48.459 INFO:tasks.workunit.client.0.smithi027.stdout:} 2020-09-12T14:26:48.459 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after v 0'0 2020-09-12T14:26:48.459 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'141 2020-09-12T14:26:48.459 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'140 2020-09-12T14:26:48.459 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'139 2020-09-12T14:26:48.460 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'138 2020-09-12T14:26:48.460 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'137 2020-09-12T14:26:48.460 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'136 2020-09-12T14:26:48.460 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'135 2020-09-12T14:26:48.460 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'134 2020-09-12T14:26:48.460 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'133 2020-09-12T14:26:48.460 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'132 2020-09-12T14:26:48.460 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'131 2020-09-12T14:26:48.461 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'130 2020-09-12T14:26:48.461 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'129 2020-09-12T14:26:48.461 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'128 2020-09-12T14:26:48.461 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'127 2020-09-12T14:26:48.461 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'126 2020-09-12T14:26:48.461 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'125 2020-09-12T14:26:48.461 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'124 2020-09-12T14:26:48.462 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'123 2020-09-12T14:26:48.462 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'122 2020-09-12T14:26:48.462 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'121 2020-09-12T14:26:48.462 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'120 2020-09-12T14:26:48.462 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'119 2020-09-12T14:26:48.462 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'118 2020-09-12T14:26:48.462 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'117 2020-09-12T14:26:48.462 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'116 2020-09-12T14:26:48.463 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'115 2020-09-12T14:26:48.463 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'114 2020-09-12T14:26:48.463 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'113 2020-09-12T14:26:48.463 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'112 2020-09-12T14:26:48.463 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'111 2020-09-12T14:26:48.463 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'110 2020-09-12T14:26:48.463 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'109 2020-09-12T14:26:48.463 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'108 2020-09-12T14:26:48.464 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'107 2020-09-12T14:26:48.464 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'106 2020-09-12T14:26:48.464 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'105 2020-09-12T14:26:48.464 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'104 2020-09-12T14:26:48.464 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'103 2020-09-12T14:26:48.464 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'102 2020-09-12T14:26:48.464 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'101 2020-09-12T14:26:48.464 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'100 2020-09-12T14:26:48.465 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'99 2020-09-12T14:26:48.465 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'98 2020-09-12T14:26:48.465 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'97 2020-09-12T14:26:48.465 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'96 2020-09-12T14:26:48.465 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'95 2020-09-12T14:26:48.465 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'94 2020-09-12T14:26:48.465 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'93 2020-09-12T14:26:48.466 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'92 2020-09-12T14:26:48.466 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'91 2020-09-12T14:26:48.466 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'90 2020-09-12T14:26:48.466 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'89 2020-09-12T14:26:48.466 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'88 2020-09-12T14:26:48.466 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'87 2020-09-12T14:26:48.466 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'86 2020-09-12T14:26:48.466 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'85 2020-09-12T14:26:48.467 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'84 2020-09-12T14:26:48.467 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'83 2020-09-12T14:26:48.467 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'82 2020-09-12T14:26:48.467 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'81 2020-09-12T14:26:48.467 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'80 2020-09-12T14:26:48.467 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'79 2020-09-12T14:26:48.467 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'78 2020-09-12T14:26:48.467 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'77 2020-09-12T14:26:48.468 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'76 2020-09-12T14:26:48.468 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'75 2020-09-12T14:26:48.468 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'74 2020-09-12T14:26:48.468 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'73 2020-09-12T14:26:48.468 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'72 2020-09-12T14:26:48.468 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'71 2020-09-12T14:26:48.468 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'70 2020-09-12T14:26:48.468 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'69 2020-09-12T14:26:48.469 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'68 2020-09-12T14:26:48.469 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'67 2020-09-12T14:26:48.469 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'66 2020-09-12T14:26:48.469 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'65 2020-09-12T14:26:48.469 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'64 2020-09-12T14:26:48.469 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'63 2020-09-12T14:26:48.469 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'62 2020-09-12T14:26:48.469 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'61 2020-09-12T14:26:48.470 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'60 2020-09-12T14:26:48.470 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'59 2020-09-12T14:26:48.470 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'58 2020-09-12T14:26:48.470 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'57 2020-09-12T14:26:48.470 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'56 2020-09-12T14:26:48.470 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'55 2020-09-12T14:26:48.470 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'54 2020-09-12T14:26:48.470 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'53 2020-09-12T14:26:48.471 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'52 2020-09-12T14:26:48.471 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'51 2020-09-12T14:26:48.471 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'50 2020-09-12T14:26:48.471 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'49 2020-09-12T14:26:48.471 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'48 2020-09-12T14:26:48.471 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'47 2020-09-12T14:26:48.471 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'46 2020-09-12T14:26:48.472 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'45 2020-09-12T14:26:48.472 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'44 2020-09-12T14:26:48.472 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'43 2020-09-12T14:26:48.472 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'42 2020-09-12T14:26:48.472 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'41 2020-09-12T14:26:48.472 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'40 2020-09-12T14:26:48.472 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'39 2020-09-12T14:26:48.472 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'38 2020-09-12T14:26:48.473 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'37 2020-09-12T14:26:48.473 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'36 2020-09-12T14:26:48.473 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'35 2020-09-12T14:26:48.473 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'34 2020-09-12T14:26:48.473 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'33 2020-09-12T14:26:48.473 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'32 2020-09-12T14:26:48.473 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'31 2020-09-12T14:26:48.473 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'30 2020-09-12T14:26:48.474 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'29 2020-09-12T14:26:48.474 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'28 2020-09-12T14:26:48.474 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'27 2020-09-12T14:26:48.474 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'26 2020-09-12T14:26:48.474 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'25 2020-09-12T14:26:48.474 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'24 2020-09-12T14:26:48.474 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'23 2020-09-12T14:26:48.474 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'22 2020-09-12T14:26:48.475 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'21 2020-09-12T14:26:48.475 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'20 2020-09-12T14:26:48.475 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'19 2020-09-12T14:26:48.475 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'18 2020-09-12T14:26:48.475 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'17 2020-09-12T14:26:48.475 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'16 2020-09-12T14:26:48.475 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'15 2020-09-12T14:26:48.475 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'14 2020-09-12T14:26:48.476 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'13 2020-09-12T14:26:48.476 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'12 2020-09-12T14:26:48.476 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'11 2020-09-12T14:26:48.476 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'10 2020-09-12T14:26:48.476 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'9 2020-09-12T14:26:48.476 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'8 2020-09-12T14:26:48.476 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'7 2020-09-12T14:26:48.476 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'6 2020-09-12T14:26:48.477 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'5 2020-09-12T14:26:48.477 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'4 2020-09-12T14:26:48.477 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'3 2020-09-12T14:26:48.477 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'2 2020-09-12T14:26:48.477 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 33'1 2020-09-12T14:26:48.477 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_up_to/copy_after earliest_dup_version 0 2020-09-12T14:26:48.477 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after v 0'0 2020-09-12T14:26:48.478 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'141 2020-09-12T14:26:48.478 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'140 2020-09-12T14:26:48.478 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'139 2020-09-12T14:26:48.478 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'138 2020-09-12T14:26:48.478 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'137 2020-09-12T14:26:48.478 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'136 2020-09-12T14:26:48.478 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'135 2020-09-12T14:26:48.478 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'134 2020-09-12T14:26:48.479 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'133 2020-09-12T14:26:48.479 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'132 2020-09-12T14:26:48.479 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'131 2020-09-12T14:26:48.479 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'130 2020-09-12T14:26:48.479 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'129 2020-09-12T14:26:48.479 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'128 2020-09-12T14:26:48.479 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'127 2020-09-12T14:26:48.479 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'126 2020-09-12T14:26:48.480 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'125 2020-09-12T14:26:48.480 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'124 2020-09-12T14:26:48.480 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'123 2020-09-12T14:26:48.480 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'122 2020-09-12T14:26:48.480 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'121 2020-09-12T14:26:48.480 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'120 2020-09-12T14:26:48.480 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'119 2020-09-12T14:26:48.480 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'118 2020-09-12T14:26:48.481 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'117 2020-09-12T14:26:48.481 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'116 2020-09-12T14:26:48.481 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'115 2020-09-12T14:26:48.481 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'114 2020-09-12T14:26:48.481 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'113 2020-09-12T14:26:48.481 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'112 2020-09-12T14:26:48.481 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'111 2020-09-12T14:26:48.481 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'110 2020-09-12T14:26:48.482 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'109 2020-09-12T14:26:48.482 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'108 2020-09-12T14:26:48.482 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'107 2020-09-12T14:26:48.482 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'106 2020-09-12T14:26:48.482 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'105 2020-09-12T14:26:48.482 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'104 2020-09-12T14:26:48.482 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'103 2020-09-12T14:26:48.483 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'102 2020-09-12T14:26:48.483 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'101 2020-09-12T14:26:48.483 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'100 2020-09-12T14:26:48.483 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'99 2020-09-12T14:26:48.483 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'98 2020-09-12T14:26:48.483 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'97 2020-09-12T14:26:48.483 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'96 2020-09-12T14:26:48.483 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'95 2020-09-12T14:26:48.484 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'94 2020-09-12T14:26:48.484 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'93 2020-09-12T14:26:48.484 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'92 2020-09-12T14:26:48.484 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'91 2020-09-12T14:26:48.484 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'90 2020-09-12T14:26:48.484 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'89 2020-09-12T14:26:48.484 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'88 2020-09-12T14:26:48.484 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'87 2020-09-12T14:26:48.485 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'86 2020-09-12T14:26:48.485 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'85 2020-09-12T14:26:48.485 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'84 2020-09-12T14:26:48.485 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'83 2020-09-12T14:26:48.485 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'82 2020-09-12T14:26:48.485 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'81 2020-09-12T14:26:48.485 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'80 2020-09-12T14:26:48.485 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'79 2020-09-12T14:26:48.486 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'78 2020-09-12T14:26:48.486 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'77 2020-09-12T14:26:48.486 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'76 2020-09-12T14:26:48.486 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'75 2020-09-12T14:26:48.486 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'74 2020-09-12T14:26:48.486 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'73 2020-09-12T14:26:48.486 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'72 2020-09-12T14:26:48.487 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'71 2020-09-12T14:26:48.487 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'70 2020-09-12T14:26:48.487 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'69 2020-09-12T14:26:48.487 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'68 2020-09-12T14:26:48.487 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'67 2020-09-12T14:26:48.487 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'66 2020-09-12T14:26:48.487 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'65 2020-09-12T14:26:48.487 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'64 2020-09-12T14:26:48.488 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'63 2020-09-12T14:26:48.488 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'62 2020-09-12T14:26:48.488 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'61 2020-09-12T14:26:48.488 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'60 2020-09-12T14:26:48.488 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'59 2020-09-12T14:26:48.488 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'58 2020-09-12T14:26:48.488 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'57 2020-09-12T14:26:48.488 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'56 2020-09-12T14:26:48.489 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'55 2020-09-12T14:26:48.489 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'54 2020-09-12T14:26:48.489 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'53 2020-09-12T14:26:48.489 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'52 2020-09-12T14:26:48.489 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'51 2020-09-12T14:26:48.489 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'50 2020-09-12T14:26:48.489 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'49 2020-09-12T14:26:48.489 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'48 2020-09-12T14:26:48.490 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'47 2020-09-12T14:26:48.490 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'46 2020-09-12T14:26:48.490 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'45 2020-09-12T14:26:48.490 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'44 2020-09-12T14:26:48.490 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'43 2020-09-12T14:26:48.490 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'42 2020-09-12T14:26:48.490 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'41 2020-09-12T14:26:48.490 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'40 2020-09-12T14:26:48.491 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'39 2020-09-12T14:26:48.491 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'38 2020-09-12T14:26:48.491 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'37 2020-09-12T14:26:48.491 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'36 2020-09-12T14:26:48.491 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'35 2020-09-12T14:26:48.491 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'34 2020-09-12T14:26:48.491 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'33 2020-09-12T14:26:48.492 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'32 2020-09-12T14:26:48.492 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'31 2020-09-12T14:26:48.492 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'30 2020-09-12T14:26:48.492 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'29 2020-09-12T14:26:48.492 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'28 2020-09-12T14:26:48.492 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'27 2020-09-12T14:26:48.492 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'26 2020-09-12T14:26:48.492 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'25 2020-09-12T14:26:48.493 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'24 2020-09-12T14:26:48.493 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'23 2020-09-12T14:26:48.493 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'22 2020-09-12T14:26:48.493 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'21 2020-09-12T14:26:48.493 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'20 2020-09-12T14:26:48.493 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'19 2020-09-12T14:26:48.493 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'18 2020-09-12T14:26:48.493 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'17 2020-09-12T14:26:48.494 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'16 2020-09-12T14:26:48.494 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'15 2020-09-12T14:26:48.494 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'14 2020-09-12T14:26:48.494 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'13 2020-09-12T14:26:48.494 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'12 2020-09-12T14:26:48.494 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'11 2020-09-12T14:26:48.494 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'10 2020-09-12T14:26:48.494 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'9 2020-09-12T14:26:48.495 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'8 2020-09-12T14:26:48.495 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'7 2020-09-12T14:26:48.495 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'6 2020-09-12T14:26:48.495 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'5 2020-09-12T14:26:48.495 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'4 2020-09-12T14:26:48.495 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'3 2020-09-12T14:26:48.495 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'2 2020-09-12T14:26:48.495 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 33'1 2020-09-12T14:26:48.496 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_up_to/copy_after earliest_dup_version 0 2020-09-12T14:26:48.496 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after v 0'0 2020-09-12T14:26:48.496 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'141 2020-09-12T14:26:48.496 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'140 2020-09-12T14:26:48.496 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'139 2020-09-12T14:26:48.496 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'138 2020-09-12T14:26:48.496 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'137 2020-09-12T14:26:48.496 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'136 2020-09-12T14:26:48.497 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'135 2020-09-12T14:26:48.497 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'134 2020-09-12T14:26:48.497 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'133 2020-09-12T14:26:48.497 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'132 2020-09-12T14:26:48.497 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'131 2020-09-12T14:26:48.497 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'130 2020-09-12T14:26:48.497 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'129 2020-09-12T14:26:48.497 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'128 2020-09-12T14:26:48.498 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'127 2020-09-12T14:26:48.498 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'126 2020-09-12T14:26:48.498 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'125 2020-09-12T14:26:48.498 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'124 2020-09-12T14:26:48.498 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'123 2020-09-12T14:26:48.498 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'122 2020-09-12T14:26:48.498 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'121 2020-09-12T14:26:48.499 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'120 2020-09-12T14:26:48.499 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'119 2020-09-12T14:26:48.499 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'118 2020-09-12T14:26:48.499 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'117 2020-09-12T14:26:48.499 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'116 2020-09-12T14:26:48.499 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'115 2020-09-12T14:26:48.499 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'114 2020-09-12T14:26:48.499 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'113 2020-09-12T14:26:48.500 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'112 2020-09-12T14:26:48.500 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'111 2020-09-12T14:26:48.500 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'110 2020-09-12T14:26:48.500 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'109 2020-09-12T14:26:48.500 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'108 2020-09-12T14:26:48.500 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'107 2020-09-12T14:26:48.500 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'106 2020-09-12T14:26:48.500 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'105 2020-09-12T14:26:48.501 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'104 2020-09-12T14:26:48.501 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'103 2020-09-12T14:26:48.501 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'102 2020-09-12T14:26:48.501 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'101 2020-09-12T14:26:48.501 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'100 2020-09-12T14:26:48.501 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'99 2020-09-12T14:26:48.501 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'98 2020-09-12T14:26:48.501 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'97 2020-09-12T14:26:48.502 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'96 2020-09-12T14:26:48.502 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'95 2020-09-12T14:26:48.502 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'94 2020-09-12T14:26:48.502 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'93 2020-09-12T14:26:48.502 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'92 2020-09-12T14:26:48.502 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'91 2020-09-12T14:26:48.502 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'90 2020-09-12T14:26:48.503 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'89 2020-09-12T14:26:48.503 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'88 2020-09-12T14:26:48.503 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'87 2020-09-12T14:26:48.503 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'86 2020-09-12T14:26:48.503 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'85 2020-09-12T14:26:48.503 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'84 2020-09-12T14:26:48.503 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'83 2020-09-12T14:26:48.503 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'82 2020-09-12T14:26:48.504 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'81 2020-09-12T14:26:48.504 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'80 2020-09-12T14:26:48.504 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'79 2020-09-12T14:26:48.504 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'78 2020-09-12T14:26:48.504 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'77 2020-09-12T14:26:48.504 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'76 2020-09-12T14:26:48.504 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'75 2020-09-12T14:26:48.504 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'74 2020-09-12T14:26:48.505 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'73 2020-09-12T14:26:48.505 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'72 2020-09-12T14:26:48.505 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'71 2020-09-12T14:26:48.505 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'70 2020-09-12T14:26:48.505 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'69 2020-09-12T14:26:48.505 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'68 2020-09-12T14:26:48.505 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'67 2020-09-12T14:26:48.505 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'66 2020-09-12T14:26:48.506 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'65 2020-09-12T14:26:48.506 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'64 2020-09-12T14:26:48.506 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'63 2020-09-12T14:26:48.506 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'62 2020-09-12T14:26:48.506 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'61 2020-09-12T14:26:48.506 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'60 2020-09-12T14:26:48.506 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'59 2020-09-12T14:26:48.506 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'58 2020-09-12T14:26:48.507 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'57 2020-09-12T14:26:48.507 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'56 2020-09-12T14:26:48.507 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'55 2020-09-12T14:26:48.507 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'54 2020-09-12T14:26:48.507 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'53 2020-09-12T14:26:48.507 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'52 2020-09-12T14:26:48.507 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'51 2020-09-12T14:26:48.507 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'50 2020-09-12T14:26:48.508 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'49 2020-09-12T14:26:48.508 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'48 2020-09-12T14:26:48.508 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'47 2020-09-12T14:26:48.508 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'46 2020-09-12T14:26:48.508 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'45 2020-09-12T14:26:48.508 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'44 2020-09-12T14:26:48.508 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'43 2020-09-12T14:26:48.509 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'42 2020-09-12T14:26:48.509 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'41 2020-09-12T14:26:48.509 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'40 2020-09-12T14:26:48.509 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'39 2020-09-12T14:26:48.509 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'38 2020-09-12T14:26:48.509 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'37 2020-09-12T14:26:48.509 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'36 2020-09-12T14:26:48.509 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'35 2020-09-12T14:26:48.510 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'34 2020-09-12T14:26:48.510 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'33 2020-09-12T14:26:48.510 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'32 2020-09-12T14:26:48.510 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'31 2020-09-12T14:26:48.510 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'30 2020-09-12T14:26:48.510 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'29 2020-09-12T14:26:48.510 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'28 2020-09-12T14:26:48.510 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'27 2020-09-12T14:26:48.511 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'26 2020-09-12T14:26:48.511 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'25 2020-09-12T14:26:48.511 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'24 2020-09-12T14:26:48.511 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'23 2020-09-12T14:26:48.511 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'22 2020-09-12T14:26:48.511 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'21 2020-09-12T14:26:48.511 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'20 2020-09-12T14:26:48.512 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'19 2020-09-12T14:26:48.512 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'18 2020-09-12T14:26:48.512 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'17 2020-09-12T14:26:48.512 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'16 2020-09-12T14:26:48.512 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'15 2020-09-12T14:26:48.512 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'14 2020-09-12T14:26:48.512 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'13 2020-09-12T14:26:48.512 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'12 2020-09-12T14:26:48.513 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'11 2020-09-12T14:26:48.513 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'10 2020-09-12T14:26:48.513 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'9 2020-09-12T14:26:48.513 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'8 2020-09-12T14:26:48.513 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'7 2020-09-12T14:26:48.513 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'6 2020-09-12T14:26:48.513 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'5 2020-09-12T14:26:48.513 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'4 2020-09-12T14:26:48.514 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'3 2020-09-12T14:26:48.514 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 34'2 2020-09-12T14:26:48.514 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_after copy log version 33'1 2020-09-12T14:26:48.514 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:02.665+0000 7f882b43f700 20 copy_up_to/copy_after earliest_dup_version 0 2020-09-12T14:26:48.514 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_after v 38'161 2020-09-12T14:26:48.514 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after earliest_dup_version 0 2020-09-12T14:26:48.514 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup version 33'1 2020-09-12T14:26:48.514 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup version 34'2 2020-09-12T14:26:48.515 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup version 34'3 2020-09-12T14:26:48.515 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup version 34'4 2020-09-12T14:26:48.515 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup version 34'5 2020-09-12T14:26:48.515 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup version 34'6 2020-09-12T14:26:48.515 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup version 34'7 2020-09-12T14:26:48.515 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup version 34'8 2020-09-12T14:26:48.515 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup version 34'9 2020-09-12T14:26:48.515 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup version 34'10 2020-09-12T14:26:48.516 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'11 2020-09-12T14:26:48.516 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'12 2020-09-12T14:26:48.516 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'13 2020-09-12T14:26:48.516 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'14 2020-09-12T14:26:48.516 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'15 2020-09-12T14:26:48.516 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'16 2020-09-12T14:26:48.516 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'17 2020-09-12T14:26:48.516 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'18 2020-09-12T14:26:48.517 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'19 2020-09-12T14:26:48.517 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'20 2020-09-12T14:26:48.517 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'21 2020-09-12T14:26:48.517 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'22 2020-09-12T14:26:48.517 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'23 2020-09-12T14:26:48.517 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'24 2020-09-12T14:26:48.517 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'25 2020-09-12T14:26:48.517 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'26 2020-09-12T14:26:48.518 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'27 2020-09-12T14:26:48.518 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'28 2020-09-12T14:26:48.518 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'29 2020-09-12T14:26:48.518 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'30 2020-09-12T14:26:48.518 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'31 2020-09-12T14:26:48.518 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'32 2020-09-12T14:26:48.518 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'33 2020-09-12T14:26:48.519 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'34 2020-09-12T14:26:48.519 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'35 2020-09-12T14:26:48.519 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'36 2020-09-12T14:26:48.519 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'37 2020-09-12T14:26:48.519 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'38 2020-09-12T14:26:48.519 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'39 2020-09-12T14:26:48.519 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'40 2020-09-12T14:26:48.519 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'41 2020-09-12T14:26:48.520 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'42 2020-09-12T14:26:48.520 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'43 2020-09-12T14:26:48.520 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'44 2020-09-12T14:26:48.520 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'45 2020-09-12T14:26:48.520 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'46 2020-09-12T14:26:48.520 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'47 2020-09-12T14:26:48.520 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'48 2020-09-12T14:26:48.520 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'49 2020-09-12T14:26:48.521 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'50 2020-09-12T14:26:48.521 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'51 2020-09-12T14:26:48.521 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'52 2020-09-12T14:26:48.521 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'53 2020-09-12T14:26:48.521 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'54 2020-09-12T14:26:48.521 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'55 2020-09-12T14:26:48.521 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'56 2020-09-12T14:26:48.521 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'57 2020-09-12T14:26:48.522 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'58 2020-09-12T14:26:48.522 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'59 2020-09-12T14:26:48.522 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'60 2020-09-12T14:26:48.522 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'61 2020-09-12T14:26:48.522 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'62 2020-09-12T14:26:48.522 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'63 2020-09-12T14:26:48.522 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'64 2020-09-12T14:26:48.522 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'65 2020-09-12T14:26:48.523 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'66 2020-09-12T14:26:48.523 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'67 2020-09-12T14:26:48.523 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'68 2020-09-12T14:26:48.523 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'69 2020-09-12T14:26:48.523 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'70 2020-09-12T14:26:48.523 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'71 2020-09-12T14:26:48.523 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'72 2020-09-12T14:26:48.523 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'73 2020-09-12T14:26:48.524 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'74 2020-09-12T14:26:48.524 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'75 2020-09-12T14:26:48.524 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'76 2020-09-12T14:26:48.524 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'77 2020-09-12T14:26:48.524 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'78 2020-09-12T14:26:48.524 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'79 2020-09-12T14:26:48.524 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'80 2020-09-12T14:26:48.525 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'81 2020-09-12T14:26:48.525 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'82 2020-09-12T14:26:48.525 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'83 2020-09-12T14:26:48.525 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'84 2020-09-12T14:26:48.525 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'85 2020-09-12T14:26:48.525 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'86 2020-09-12T14:26:48.525 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'87 2020-09-12T14:26:48.525 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'88 2020-09-12T14:26:48.526 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'89 2020-09-12T14:26:48.526 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'90 2020-09-12T14:26:48.526 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'91 2020-09-12T14:26:48.526 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'92 2020-09-12T14:26:48.526 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'93 2020-09-12T14:26:48.526 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'94 2020-09-12T14:26:48.526 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'95 2020-09-12T14:26:48.526 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'96 2020-09-12T14:26:48.527 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'97 2020-09-12T14:26:48.527 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'98 2020-09-12T14:26:48.527 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'99 2020-09-12T14:26:48.527 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'100 2020-09-12T14:26:48.527 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'101 2020-09-12T14:26:48.527 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'102 2020-09-12T14:26:48.527 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'103 2020-09-12T14:26:48.527 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'104 2020-09-12T14:26:48.528 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'105 2020-09-12T14:26:48.528 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'106 2020-09-12T14:26:48.528 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'107 2020-09-12T14:26:48.528 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'108 2020-09-12T14:26:48.528 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'109 2020-09-12T14:26:48.528 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'110 2020-09-12T14:26:48.528 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'111 2020-09-12T14:26:48.528 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'112 2020-09-12T14:26:48.529 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'113 2020-09-12T14:26:48.529 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'114 2020-09-12T14:26:48.529 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'115 2020-09-12T14:26:48.529 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'116 2020-09-12T14:26:48.529 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'117 2020-09-12T14:26:48.529 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'118 2020-09-12T14:26:48.529 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'119 2020-09-12T14:26:48.530 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'120 2020-09-12T14:26:48.530 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'121 2020-09-12T14:26:48.530 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'122 2020-09-12T14:26:48.530 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'123 2020-09-12T14:26:48.530 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'124 2020-09-12T14:26:48.530 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'125 2020-09-12T14:26:48.530 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'126 2020-09-12T14:26:48.530 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'127 2020-09-12T14:26:48.531 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'128 2020-09-12T14:26:48.531 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'129 2020-09-12T14:26:48.531 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'130 2020-09-12T14:26:48.531 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'131 2020-09-12T14:26:48.531 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'132 2020-09-12T14:26:48.531 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'133 2020-09-12T14:26:48.531 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'134 2020-09-12T14:26:48.531 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'135 2020-09-12T14:26:48.532 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'136 2020-09-12T14:26:48.532 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'137 2020-09-12T14:26:48.532 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'138 2020-09-12T14:26:48.532 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'139 2020-09-12T14:26:48.532 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'140 2020-09-12T14:26:48.532 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 34'141 2020-09-12T14:26:48.532 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 38'142 2020-09-12T14:26:48.532 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 38'143 2020-09-12T14:26:48.533 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 38'144 2020-09-12T14:26:48.533 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 38'145 2020-09-12T14:26:48.533 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 38'146 2020-09-12T14:26:48.533 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 38'147 2020-09-12T14:26:48.533 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 38'148 2020-09-12T14:26:48.533 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 38'149 2020-09-12T14:26:48.533 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 38'150 2020-09-12T14:26:48.533 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 38'151 2020-09-12T14:26:48.534 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 38'152 2020-09-12T14:26:48.534 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 38'153 2020-09-12T14:26:48.534 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 38'154 2020-09-12T14:26:48.534 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 38'155 2020-09-12T14:26:48.534 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 38'156 2020-09-12T14:26:48.534 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 38'157 2020-09-12T14:26:48.534 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 38'158 2020-09-12T14:26:48.534 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 38'159 2020-09-12T14:26:48.535 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 38'160 2020-09-12T14:26:48.535 INFO:tasks.workunit.client.0.smithi027.stdout:td/osd-backfill-recovery-log/osd.0.log:2020-09-12T14:26:37.161+0000 7f882b43f700 20 copy_up_to/copy_after copy dup from log version 38'161 2020-09-12T14:26:48.535 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:84: _common_test: jq '.pg_log_t.log | length' td/osd-backfill-recovery-log/result.log 2020-09-12T14:26:48.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:84: _common_test: LOGLEN=151 2020-09-12T14:26:48.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:85: _common_test: '[' 151 '!=' 151 ']' 2020-09-12T14:26:48.535 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:89: _common_test: jq '.pg_log_t.dups | length' td/osd-backfill-recovery-log/result.log 2020-09-12T14:26:48.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:89: _common_test: DUPSLEN=10 2020-09-12T14:26:48.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:90: _common_test: '[' 10 '!=' 10 ']' 2020-09-12T14:26:48.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:94: _common_test: grep 'copy_up_to\|copy_after' td/osd-backfill-recovery-log/osd.0.log td/osd-backfill-recovery-log/osd.1.log td/osd-backfill-recovery-log/osd.2.log td/osd-backfill-recovery-log/osd.3.log td/osd-backfill-recovery-log/osd.4.log td/osd-backfill-recovery-log/osd.5.log 2020-09-12T14:26:48.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:95: _common_test: rm -f td/osd-backfill-recovery-log/result.log 2020-09-12T14:26:48.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:96: _common_test: '[' 0 '!=' 0 ']' 2020-09-12T14:26:48.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-recovery-log.sh:34: run: teardown td/osd-backfill-recovery-log 2020-09-12T14:26:48.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-backfill-recovery-log 2020-09-12T14:26:48.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:26:48.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-backfill-recovery-log KILL 2020-09-12T14:26:48.537 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:26:48.537 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:26:48.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:26:48.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:26:48.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:26:48.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:26:48.537 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:26:48.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:26:48.538 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:26:48.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:26:48.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:26:48.538 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:26:48.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:26:48.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:26:48.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:26:48.539 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:26:48.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:26:48.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:26:48.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-backfill-recovery-log 2020-09-12T14:26:48.539 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:26:48.539 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:26:48.539 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:26:48.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.98804 2020-09-12T14:26:48.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:26:48.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:26:48.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2151: main: code=0 2020-09-12T14:26:48.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2155: main: teardown td/osd-backfill-recovery-log 0 2020-09-12T14:26:48.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-backfill-recovery-log 2020-09-12T14:26:48.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2020-09-12T14:26:48.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-backfill-recovery-log KILL 2020-09-12T14:26:48.541 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:26:48.541 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:26:48.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:26:48.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:26:48.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:26:48.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:26:48.541 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:26:48.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:26:48.542 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:26:48.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:26:48.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:26:48.542 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:26:48.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:26:48.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:26:48.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:26:48.543 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:26:48.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:26:48.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2020-09-12T14:26:48.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-backfill-recovery-log 2020-09-12T14:26:48.543 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:26:48.543 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:26:48.543 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.98804 2020-09-12T14:26:48.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.98804 2020-09-12T14:26:48.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:26:48.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:26:48.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2156: main: return 0 2020-09-12T14:26:48.544 INFO:teuthology.orchestra.run:Running command with timeout 3600 2020-09-12T14:26:48.545 INFO:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2020-09-12T14:26:48.590 INFO:tasks.workunit:Running workunit osd/osd-backfill-space.sh... 2020-09-12T14:26:48.591 INFO:teuthology.orchestra.run.smithi027:workunit test osd/osd-backfill-space.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f346cc73d15a568d3bec1c99a58953de76f57b85 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh 2020-09-12T14:26:48.613 INFO:tasks.workunit.client.0.smithi027.stderr:stty: 'standard input': Inappropriate ioctl for device 2020-09-12T14:26:48.623 INFO:tasks.workunit.client.0.smithi027.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2020-09-12T14:26:48.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2020-09-12T14:26:48.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2020-09-12T14:26:48.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2145: main: export PYTHONWARNINGS=ignore 2020-09-12T14:26:48.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2145: main: PYTHONWARNINGS=ignore 2020-09-12T14:26:48.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2146: main: export CEPH_CONF=/dev/null 2020-09-12T14:26:48.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2146: main: CEPH_CONF=/dev/null 2020-09-12T14:26:48.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2147: main: unset CEPH_ARGS 2020-09-12T14:26:48.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2149: main: local code 2020-09-12T14:26:48.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2150: main: run td/osd-backfill-space 2020-09-12T14:26:48.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:21: run: local dir=td/osd-backfill-space 2020-09-12T14:26:48.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:22: run: shift 2020-09-12T14:26:48.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:24: run: export CEPH_MON=127.0.0.1:7180 2020-09-12T14:26:48.625 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:24: run: CEPH_MON=127.0.0.1:7180 2020-09-12T14:26:48.625 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:25: run: export CEPH_ARGS 2020-09-12T14:26:48.625 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:26: run: uuidgen 2020-09-12T14:26:48.626 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:26: run: CEPH_ARGS+='--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none ' 2020-09-12T14:26:48.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:27: run: CEPH_ARGS+='--mon-host=127.0.0.1:7180 ' 2020-09-12T14:26:48.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:28: run: CEPH_ARGS+='--osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 ' 2020-09-12T14:26:48.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:29: run: CEPH_ARGS+='--fake_statfs_for_testing=3686400 ' 2020-09-12T14:26:48.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:30: run: CEPH_ARGS+='--osd_max_backfills=10 ' 2020-09-12T14:26:48.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:31: run: export objects=600 2020-09-12T14:26:48.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:31: run: objects=600 2020-09-12T14:26:48.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:32: run: export poolprefix=test 2020-09-12T14:26:48.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:32: run: poolprefix=test 2020-09-12T14:26:48.628 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:34: run: set 2020-09-12T14:26:48.628 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:34: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2020-09-12T14:26:48.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:34: run: local 'funcs=TEST_backfill_grow 2020-09-12T14:26:48.628 INFO:tasks.workunit.client.0.smithi027.stderr:TEST_backfill_multi_partial 2020-09-12T14:26:48.628 INFO:tasks.workunit.client.0.smithi027.stderr:TEST_backfill_test_multi 2020-09-12T14:26:48.628 INFO:tasks.workunit.client.0.smithi027.stderr:TEST_backfill_test_sametarget 2020-09-12T14:26:48.628 INFO:tasks.workunit.client.0.smithi027.stderr:TEST_backfill_test_simple 2020-09-12T14:26:48.629 INFO:tasks.workunit.client.0.smithi027.stderr:TEST_ec_backfill_grow 2020-09-12T14:26:48.629 INFO:tasks.workunit.client.0.smithi027.stderr:TEST_ec_backfill_multi 2020-09-12T14:26:48.629 INFO:tasks.workunit.client.0.smithi027.stderr:TEST_ec_backfill_simple' 2020-09-12T14:26:48.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:35: run: for func in $funcs 2020-09-12T14:26:48.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:36: run: setup td/osd-backfill-space 2020-09-12T14:26:48.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-backfill-space 2020-09-12T14:26:48.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-backfill-space 2020-09-12T14:26:48.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-backfill-space 2020-09-12T14:26:48.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:26:48.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-backfill-space KILL 2020-09-12T14:26:48.630 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:26:48.630 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:26:48.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:26:48.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:26:48.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:26:48.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:26:48.631 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:26:48.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:26:48.632 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:26:48.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:26:48.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:26:48.632 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:26:48.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:26:48.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:26:48.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:26:48.633 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:26:48.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:26:48.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:26:48.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-backfill-space 2020-09-12T14:26:48.635 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:26:48.635 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:26:48.635 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:26:48.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.145692 2020-09-12T14:26:48.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:26:48.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:26:48.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-backfill-space 2020-09-12T14:26:48.637 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2020-09-12T14:26:48.637 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:26:48.637 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:26:48.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.145692 2020-09-12T14:26:48.638 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2020-09-12T14:26:48.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2020-09-12T14:26:48.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2020-09-12T14:26:48.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2020-09-12T14:26:48.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-backfill-space 1' TERM HUP INT 2020-09-12T14:26:48.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:37: run: TEST_backfill_grow td/osd-backfill-space 2020-09-12T14:26:48.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:557: TEST_backfill_grow: local dir=td/osd-backfill-space 2020-09-12T14:26:48.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:558: TEST_backfill_grow: local poolname=test 2020-09-12T14:26:48.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:559: TEST_backfill_grow: local OSDS=3 2020-09-12T14:26:48.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:561: TEST_backfill_grow: run_mon td/osd-backfill-space a 2020-09-12T14:26:48.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-backfill-space 2020-09-12T14:26:48.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2020-09-12T14:26:48.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2020-09-12T14:26:48.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2020-09-12T14:26:48.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-backfill-space/a 2020-09-12T14:26:48.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-backfill-space/a --run-dir=td/osd-backfill-space 2020-09-12T14:26:48.950 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2020-09-12T14:26:48.950 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:26:48.951 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:26:48.951 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:26:48.951 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:26:48.951 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:26:48.951 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:26:48.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-backfill-space/a '--log-file=td/osd-backfill-space/$name.log' '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --mon-cluster-log-file=td/osd-backfill-space/log --run-dir=td/osd-backfill-space '--pid-file=td/osd-backfill-space/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 2020-09-12T14:26:49.110 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: cat 2020-09-12T14:26:49.111 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a fsid 2020-09-12T14:26:49.111 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:26:49.112 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:26:49.112 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=fsid 2020-09-12T14:26:49.112 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .fsid 2020-09-12T14:26:49.112 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:26:49.112 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:26:49.112 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:26:49.113 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:26:49.113 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:26:49.113 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:26:49.113 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.145692/ceph-mon.a.asok 2020-09-12T14:26:49.114 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:26:49.114 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.145692/ceph-mon.a.asok config get fsid 2020-09-12T14:26:49.220 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a mon_host 2020-09-12T14:26:49.221 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:26:49.221 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:26:49.221 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=mon_host 2020-09-12T14:26:49.221 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .mon_host 2020-09-12T14:26:49.221 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:26:49.221 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:26:49.221 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:26:49.222 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:26:49.222 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:26:49.223 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:26:49.223 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.145692/ceph-mon.a.asok 2020-09-12T14:26:49.223 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:26:49.223 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.145692/ceph-mon.a.asok config get mon_host 2020-09-12T14:26:49.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:562: TEST_backfill_grow: run_mgr td/osd-backfill-space x 2020-09-12T14:26:49.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: local dir=td/osd-backfill-space 2020-09-12T14:26:49.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: shift 2020-09-12T14:26:49.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: local id=x 2020-09-12T14:26:49.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: shift 2020-09-12T14:26:49.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: local data=td/osd-backfill-space/x 2020-09-12T14:26:49.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:558: run_mgr: ceph config set mgr mgr/devicehealth/enable_monitoring off --force 2020-09-12T14:26:49.844 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: get_asok_path 2020-09-12T14:26:49.845 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:26:49.845 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:26:49.845 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:26:49.845 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:26:49.845 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:26:49.845 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:26:49.846 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:26:49.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-backfill-space/x '--log-file=td/osd-backfill-space/$name.log' '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --run-dir=td/osd-backfill-space '--pid-file=td/osd-backfill-space/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:26:49.870 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:564: TEST_backfill_grow: expr 3 - 1 2020-09-12T14:26:49.871 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:564: TEST_backfill_grow: seq 0 2 2020-09-12T14:26:49.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:564: TEST_backfill_grow: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:26:49.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:566: TEST_backfill_grow: run_osd td/osd-backfill-space 0 2020-09-12T14:26:49.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:26:49.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:26:49.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=0 2020-09-12T14:26:49.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:26:49.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/0 2020-09-12T14:26:49.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:26:49.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:26:49.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:26:49.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:26:49.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/0' 2020-09-12T14:26:49.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/0/journal' 2020-09-12T14:26:49.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:26:49.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:26:49.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:26:49.876 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:26:49.876 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:26:49.877 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:26:49.877 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:26:49.877 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:26:49.877 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:26:49.877 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:26:49.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:26:49.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:26:49.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:26:49.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:26:49.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:26:49.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:26:49.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:26:49.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:26:49.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:26:49.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:26:49.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:26:49.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/0 2020-09-12T14:26:49.879 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:26:49.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=eaea4ad0-08c4-4433-ada9-0903fe013c91 2020-09-12T14:26:49.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd0 eaea4ad0-08c4-4433-ada9-0903fe013c91' 2020-09-12T14:26:49.880 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:26:49.880 INFO:tasks.workunit.client.0.smithi027.stdout:add osd0 eaea4ad0-08c4-4433-ada9-0903fe013c91 2020-09-12T14:26:49.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQCp2lxfpA/JNRAAnbxR7jQ5rKoUQBXH3Sr0Jw== 2020-09-12T14:26:49.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQCp2lxfpA/JNRAAnbxR7jQ5rKoUQBXH3Sr0Jw=="}' 2020-09-12T14:26:49.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new eaea4ad0-08c4-4433-ada9-0903fe013c91 -i td/osd-backfill-space/0/new.json 2020-09-12T14:26:50.258 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:26:50.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/0/new.json 2020-09-12T14:26:50.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 0 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/0 --osd-journal=td/osd-backfill-space/0/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQCp2lxfpA/JNRAAnbxR7jQ5rKoUQBXH3Sr0Jw== --osd-uuid eaea4ad0-08c4-4433-ada9-0903fe013c91 2020-09-12T14:26:50.297 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:26:50.296+0000 7f4436addec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:26:50.298 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:26:50.298+0000 7f4436addec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:26:50.300 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:26:50.299+0000 7f4436addec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:26:50.301 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:26:50.301+0000 7f4436addec0 -1 bluestore(td/osd-backfill-space/0/block) _read_bdev_label failed to open td/osd-backfill-space/0/block: (2) No such file or directory 2020-09-12T14:26:50.301 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:26:50.301+0000 7f4436addec0 -1 bluestore(td/osd-backfill-space/0/block) _read_bdev_label failed to open td/osd-backfill-space/0/block: (2) No such file or directory 2020-09-12T14:26:50.301 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:26:50.301+0000 7f4436addec0 -1 bluestore(td/osd-backfill-space/0/block) _read_bdev_label failed to open td/osd-backfill-space/0/block: (2) No such file or directory 2020-09-12T14:26:50.352 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:26:50.352+0000 7f4436addec0 -1 bluestore(td/osd-backfill-space/0) _read_fsid unparsable uuid 2020-09-12T14:26:54.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/0/keyring 2020-09-12T14:26:54.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:26:54.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd0 key to auth repository 2020-09-12T14:26:54.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:26:54.575 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd0 key to auth repository 2020-09-12T14:26:55.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.0 2020-09-12T14:26:55.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:26:55.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 0 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/0 --osd-journal=td/osd-backfill-space/0/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:26:55.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:26:55.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:26:55.130 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.0 2020-09-12T14:26:55.154 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:26:55.154+0000 7f2bc627dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:26:55.174 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:26:55.174+0000 7f2bc627dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:26:55.175 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:26:55.175+0000 7f2bc627dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:26:55.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 0 2020-09-12T14:26:55.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:26:55.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T14:26:55.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:26:55.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:26:55.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:26:55.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:26:55.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:26:55.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:26:55.539 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:26:55.699 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:26:55.699+0000 7f2bc627dec0 -1 Falling back to public interface 2020-09-12T14:26:55.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:26:56.500 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:26:56.499+0000 7f2bc627dec0 -1 osd.0 0 log_to_monitors {default=true} 2020-09-12T14:26:56.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:26:56.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:26:56.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:26:56.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:26:56.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:26:56.957 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:26:57.392 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/146561,v1:127.0.0.1:6803/146561] [v2:127.0.0.1:6804/146561,v1:127.0.0.1:6805/146561] exists,up eaea4ad0-08c4-4433-ada9-0903fe013c91 2020-09-12T14:26:57.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:26:57.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:26:57.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:26:57.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:564: TEST_backfill_grow: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:26:57.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:566: TEST_backfill_grow: run_osd td/osd-backfill-space 1 2020-09-12T14:26:57.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:26:57.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:26:57.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=1 2020-09-12T14:26:57.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:26:57.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/1 2020-09-12T14:26:57.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:26:57.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:26:57.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:26:57.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:26:57.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/1' 2020-09-12T14:26:57.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/1/journal' 2020-09-12T14:26:57.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:26:57.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:26:57.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:26:57.395 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:26:57.396 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:26:57.396 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:26:57.396 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:26:57.396 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:26:57.396 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:26:57.396 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:26:57.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:26:57.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:26:57.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:26:57.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:26:57.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:26:57.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:26:57.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:26:57.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:26:57.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:26:57.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:26:57.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:26:57.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/1 2020-09-12T14:26:57.398 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:26:57.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=1e18252f-b630-46ba-b60e-b549cf55f880 2020-09-12T14:26:57.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd1 1e18252f-b630-46ba-b60e-b549cf55f880' 2020-09-12T14:26:57.399 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:26:57.399 INFO:tasks.workunit.client.0.smithi027.stdout:add osd1 1e18252f-b630-46ba-b60e-b549cf55f880 2020-09-12T14:26:57.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQCx2lxfR8JxGBAAlvK93xTs1OZQLp05uy3dYg== 2020-09-12T14:26:57.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQCx2lxfR8JxGBAAlvK93xTs1OZQLp05uy3dYg=="}' 2020-09-12T14:26:57.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 1e18252f-b630-46ba-b60e-b549cf55f880 -i td/osd-backfill-space/1/new.json 2020-09-12T14:26:57.947 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:26:57.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/1/new.json 2020-09-12T14:26:57.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 1 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/1 --osd-journal=td/osd-backfill-space/1/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQCx2lxfR8JxGBAAlvK93xTs1OZQLp05uy3dYg== --osd-uuid 1e18252f-b630-46ba-b60e-b549cf55f880 2020-09-12T14:26:57.986 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:26:57.986+0000 7f9878676ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:26:57.988 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:26:57.988+0000 7f9878676ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:26:57.989 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:26:57.989+0000 7f9878676ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:26:57.989 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:26:57.989+0000 7f9878676ec0 -1 bluestore(td/osd-backfill-space/1/block) _read_bdev_label failed to open td/osd-backfill-space/1/block: (2) No such file or directory 2020-09-12T14:26:57.990 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:26:57.989+0000 7f9878676ec0 -1 bluestore(td/osd-backfill-space/1/block) _read_bdev_label failed to open td/osd-backfill-space/1/block: (2) No such file or directory 2020-09-12T14:26:57.990 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:26:57.989+0000 7f9878676ec0 -1 bluestore(td/osd-backfill-space/1/block) _read_bdev_label failed to open td/osd-backfill-space/1/block: (2) No such file or directory 2020-09-12T14:26:58.040 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:26:58.039+0000 7f9878676ec0 -1 bluestore(td/osd-backfill-space/1) _read_fsid unparsable uuid 2020-09-12T14:27:02.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/1/keyring 2020-09-12T14:27:02.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:27:02.029 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd1 key to auth repository 2020-09-12T14:27:02.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd1 key to auth repository 2020-09-12T14:27:02.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:27:02.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.1 2020-09-12T14:27:02.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:27:02.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 1 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/1 --osd-journal=td/osd-backfill-space/1/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:27:02.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:27:02.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:27:02.580 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T14:27:02.604 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:27:02.604+0000 7f553b85dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:27:02.620 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:27:02.619+0000 7f553b85dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:27:02.622 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:27:02.621+0000 7f553b85dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:27:02.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 1 2020-09-12T14:27:02.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:27:02.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:27:02.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:27:02.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:27:02.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:27:02.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:27:02.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:27:02.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:27:02.984 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:27:03.141 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:27:03.140+0000 7f553b85dec0 -1 Falling back to public interface 2020-09-12T14:27:03.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:27:03.954 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:27:03.954+0000 7f553b85dec0 -1 osd.1 0 log_to_monitors {default=true} 2020-09-12T14:27:04.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:27:04.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:27:04.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:27:04.393 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:27:04.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:27:04.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:27:04.823 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/147555,v1:127.0.0.1:6811/147555] [v2:127.0.0.1:6812/147555,v1:127.0.0.1:6813/147555] exists,up 1e18252f-b630-46ba-b60e-b549cf55f880 2020-09-12T14:27:04.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:27:04.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:27:04.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:27:04.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:564: TEST_backfill_grow: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:27:04.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:566: TEST_backfill_grow: run_osd td/osd-backfill-space 2 2020-09-12T14:27:04.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:27:04.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:27:04.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=2 2020-09-12T14:27:04.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:27:04.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/2 2020-09-12T14:27:04.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:27:04.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:27:04.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:27:04.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:27:04.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/2' 2020-09-12T14:27:04.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/2/journal' 2020-09-12T14:27:04.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:27:04.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:27:04.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:27:04.826 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:27:04.826 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:27:04.827 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:27:04.828 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:27:04.828 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:27:04.828 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:27:04.829 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:27:04.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:27:04.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:27:04.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:27:04.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:27:04.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:27:04.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:27:04.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:27:04.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:27:04.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:27:04.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:27:04.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:27:04.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/2 2020-09-12T14:27:04.830 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:27:04.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=d2d07e13-2054-4be2-8c53-5658d8ad4949 2020-09-12T14:27:04.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd2 d2d07e13-2054-4be2-8c53-5658d8ad4949' 2020-09-12T14:27:04.831 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:27:04.831 INFO:tasks.workunit.client.0.smithi027.stdout:add osd2 d2d07e13-2054-4be2-8c53-5658d8ad4949 2020-09-12T14:27:04.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQC42lxffPJTMhAAaXilKQ0scSs26m5x0DN8uQ== 2020-09-12T14:27:04.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQC42lxffPJTMhAAaXilKQ0scSs26m5x0DN8uQ=="}' 2020-09-12T14:27:04.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new d2d07e13-2054-4be2-8c53-5658d8ad4949 -i td/osd-backfill-space/2/new.json 2020-09-12T14:27:05.392 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:27:05.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/2/new.json 2020-09-12T14:27:05.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 2 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/2 --osd-journal=td/osd-backfill-space/2/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQC42lxffPJTMhAAaXilKQ0scSs26m5x0DN8uQ== --osd-uuid d2d07e13-2054-4be2-8c53-5658d8ad4949 2020-09-12T14:27:05.432 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:27:05.431+0000 7f35b7ec2ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:27:05.433 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:27:05.433+0000 7f35b7ec2ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:27:05.435 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:27:05.434+0000 7f35b7ec2ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:27:05.436 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:27:05.435+0000 7f35b7ec2ec0 -1 bluestore(td/osd-backfill-space/2/block) _read_bdev_label failed to open td/osd-backfill-space/2/block: (2) No such file or directory 2020-09-12T14:27:05.436 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:27:05.435+0000 7f35b7ec2ec0 -1 bluestore(td/osd-backfill-space/2/block) _read_bdev_label failed to open td/osd-backfill-space/2/block: (2) No such file or directory 2020-09-12T14:27:05.437 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:27:05.435+0000 7f35b7ec2ec0 -1 bluestore(td/osd-backfill-space/2/block) _read_bdev_label failed to open td/osd-backfill-space/2/block: (2) No such file or directory 2020-09-12T14:27:05.528 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:27:05.528+0000 7f35b7ec2ec0 -1 bluestore(td/osd-backfill-space/2) _read_fsid unparsable uuid 2020-09-12T14:27:09.767 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/2/keyring 2020-09-12T14:27:09.767 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:27:09.768 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd2 key to auth repository 2020-09-12T14:27:09.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd2 key to auth repository 2020-09-12T14:27:09.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:27:10.328 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.2 2020-09-12T14:27:10.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.2 2020-09-12T14:27:10.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 2 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/2 --osd-journal=td/osd-backfill-space/2/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:27:10.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:27:10.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:27:10.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:27:10.353 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:27:10.352+0000 7f4ec83d6ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:27:10.367 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:27:10.366+0000 7f4ec83d6ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:27:10.368 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:27:10.367+0000 7f4ec83d6ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:27:10.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 2 2020-09-12T14:27:10.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:27:10.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T14:27:10.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:27:10.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:27:10.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:27:10.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:27:10.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:27:10.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:27:10.740 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:27:10.895 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:27:10.893+0000 7f4ec83d6ec0 -1 Falling back to public interface 2020-09-12T14:27:11.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:27:11.734 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:27:11.732+0000 7f4ec83d6ec0 -1 osd.2 0 log_to_monitors {default=true} 2020-09-12T14:27:12.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:27:12.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:27:12.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:27:12.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:27:12.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:27:12.163 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:27:12.584 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/148549,v1:127.0.0.1:6819/148549] [v2:127.0.0.1:6820/148549,v1:127.0.0.1:6821/148549] exists,up d2d07e13-2054-4be2-8c53-5658d8ad4949 2020-09-12T14:27:12.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:27:12.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:27:12.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:27:12.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:569: TEST_backfill_grow: ceph osd set-backfillfull-ratio .85 2020-09-12T14:27:13.320 INFO:tasks.workunit.client.0.smithi027.stderr:osd set-backfillfull-ratio 0.85 2020-09-12T14:27:13.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:571: TEST_backfill_grow: create_pool test 1 1 2020-09-12T14:27:13.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test 1 1 2020-09-12T14:27:13.961 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test' already exists 2020-09-12T14:27:13.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:27:14.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:572: TEST_backfill_grow: ceph osd pool set test size 3 2020-09-12T14:27:15.738 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 1 size to 3 2020-09-12T14:27:15.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:573: TEST_backfill_grow: sleep 5 2020-09-12T14:27:20.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:575: TEST_backfill_grow: wait_for_clean 2020-09-12T14:27:20.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:27:20.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:27:20.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:27:20.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:27:20.750 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 90 .1 2020-09-12T14:27:20.750 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:27:20.751 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:27:20.751 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:27:20.751 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:27:20.751 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:27:20.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:27:20.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:27:20.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:27:20.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:27:20.858 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:27:21.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:27:21.266 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:27:21.267 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T14:27:21.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:27:21.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:27:21.267 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:27:21.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836487 2020-09-12T14:27:21.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836487 2020-09-12T14:27:21.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836487' 2020-09-12T14:27:21.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:27:21.415 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:27:21.558 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672966 2020-09-12T14:27:21.558 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672966 2020-09-12T14:27:21.558 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836487 1-42949672966' 2020-09-12T14:27:21.558 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:27:21.559 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:27:21.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509444 2020-09-12T14:27:21.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509444 2020-09-12T14:27:21.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836487 1-42949672966 2-64424509444' 2020-09-12T14:27:21.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:27:21.724 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836487 2020-09-12T14:27:21.724 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:27:21.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:27:21.725 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836487 2020-09-12T14:27:21.725 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:27:21.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836487 2020-09-12T14:27:21.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2020-09-12T14:27:21.726 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836487 2020-09-12T14:27:21.727 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:27:22.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836487 -lt 21474836487 2020-09-12T14:27:22.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:27:22.133 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672966 2020-09-12T14:27:22.133 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:27:22.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:27:22.135 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672966 2020-09-12T14:27:22.135 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:27:22.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672966 2020-09-12T14:27:22.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672966' 2020-09-12T14:27:22.137 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672966 2020-09-12T14:27:22.137 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:27:22.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672966 -lt 42949672966 2020-09-12T14:27:22.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:27:22.563 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509444 2020-09-12T14:27:22.563 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:27:22.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:27:22.565 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509444 2020-09-12T14:27:22.565 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:27:22.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509444 2020-09-12T14:27:22.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509444' 2020-09-12T14:27:22.567 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509444 2020-09-12T14:27:22.568 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:27:22.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509442 -lt 64424509444 2020-09-12T14:27:22.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:27:23.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:27:23.977 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:27:24.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509445 -lt 64424509444 2020-09-12T14:27:24.398 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:27:24.398 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:27:24.399 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:27:24.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 1 == 0 2020-09-12T14:27:24.928 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:27:24.929 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:27:24.929 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:27:24.929 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:27:24.929 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:27:24.929 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:27:24.929 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:27:25.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:27:25.318 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:27:25.319 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:27:25.319 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:27:25.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 1 2020-09-12T14:27:25.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T14:27:25.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T14:27:25.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:577: TEST_backfill_grow: dd if=/dev/urandom of=td/osd-backfill-space/4kdata bs=1k count=4 2020-09-12T14:27:25.870 INFO:tasks.workunit.client.0.smithi027.stderr:4+0 records in 2020-09-12T14:27:25.870 INFO:tasks.workunit.client.0.smithi027.stderr:4+0 records out 2020-09-12T14:27:25.870 INFO:tasks.workunit.client.0.smithi027.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 6.8924e-05 s, 59.4 MB/s 2020-09-12T14:27:25.870 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: seq 1 600 2020-09-12T14:27:25.871 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:25.871 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj1 td/osd-backfill-space/4kdata 2020-09-12T14:27:26.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:26.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj2 td/osd-backfill-space/4kdata 2020-09-12T14:27:26.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:26.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj3 td/osd-backfill-space/4kdata 2020-09-12T14:27:26.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:26.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj4 td/osd-backfill-space/4kdata 2020-09-12T14:27:26.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:26.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj5 td/osd-backfill-space/4kdata 2020-09-12T14:27:26.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:26.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj6 td/osd-backfill-space/4kdata 2020-09-12T14:27:26.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:26.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj7 td/osd-backfill-space/4kdata 2020-09-12T14:27:26.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:26.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj8 td/osd-backfill-space/4kdata 2020-09-12T14:27:26.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:26.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj9 td/osd-backfill-space/4kdata 2020-09-12T14:27:27.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:27.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj10 td/osd-backfill-space/4kdata 2020-09-12T14:27:27.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:27.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj11 td/osd-backfill-space/4kdata 2020-09-12T14:27:27.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:27.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj12 td/osd-backfill-space/4kdata 2020-09-12T14:27:27.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:27.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj13 td/osd-backfill-space/4kdata 2020-09-12T14:27:27.552 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:27.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj14 td/osd-backfill-space/4kdata 2020-09-12T14:27:27.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:27.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj15 td/osd-backfill-space/4kdata 2020-09-12T14:27:27.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:27.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj16 td/osd-backfill-space/4kdata 2020-09-12T14:27:27.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:27.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj17 td/osd-backfill-space/4kdata 2020-09-12T14:27:28.089 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:28.090 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj18 td/osd-backfill-space/4kdata 2020-09-12T14:27:28.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:28.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj19 td/osd-backfill-space/4kdata 2020-09-12T14:27:28.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:28.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj20 td/osd-backfill-space/4kdata 2020-09-12T14:27:28.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:28.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj21 td/osd-backfill-space/4kdata 2020-09-12T14:27:28.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:28.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj22 td/osd-backfill-space/4kdata 2020-09-12T14:27:28.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:28.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj23 td/osd-backfill-space/4kdata 2020-09-12T14:27:28.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:28.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj24 td/osd-backfill-space/4kdata 2020-09-12T14:27:28.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:28.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj25 td/osd-backfill-space/4kdata 2020-09-12T14:27:29.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:29.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj26 td/osd-backfill-space/4kdata 2020-09-12T14:27:29.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:29.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj27 td/osd-backfill-space/4kdata 2020-09-12T14:27:29.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:29.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj28 td/osd-backfill-space/4kdata 2020-09-12T14:27:29.481 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:29.481 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj29 td/osd-backfill-space/4kdata 2020-09-12T14:27:29.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:29.616 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj30 td/osd-backfill-space/4kdata 2020-09-12T14:27:29.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:29.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj31 td/osd-backfill-space/4kdata 2020-09-12T14:27:29.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:29.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj32 td/osd-backfill-space/4kdata 2020-09-12T14:27:30.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:30.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj33 td/osd-backfill-space/4kdata 2020-09-12T14:27:30.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:30.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj34 td/osd-backfill-space/4kdata 2020-09-12T14:27:30.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:30.261 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj35 td/osd-backfill-space/4kdata 2020-09-12T14:27:30.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:30.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj36 td/osd-backfill-space/4kdata 2020-09-12T14:27:30.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:30.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj37 td/osd-backfill-space/4kdata 2020-09-12T14:27:30.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:30.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj38 td/osd-backfill-space/4kdata 2020-09-12T14:27:30.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:30.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj39 td/osd-backfill-space/4kdata 2020-09-12T14:27:30.914 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:30.915 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj40 td/osd-backfill-space/4kdata 2020-09-12T14:27:31.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:31.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj41 td/osd-backfill-space/4kdata 2020-09-12T14:27:31.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:31.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj42 td/osd-backfill-space/4kdata 2020-09-12T14:27:31.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:31.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj43 td/osd-backfill-space/4kdata 2020-09-12T14:27:31.417 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:31.417 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj44 td/osd-backfill-space/4kdata 2020-09-12T14:27:31.551 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:31.552 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj45 td/osd-backfill-space/4kdata 2020-09-12T14:27:31.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:31.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj46 td/osd-backfill-space/4kdata 2020-09-12T14:27:31.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:31.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj47 td/osd-backfill-space/4kdata 2020-09-12T14:27:31.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:31.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj48 td/osd-backfill-space/4kdata 2020-09-12T14:27:32.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:32.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj49 td/osd-backfill-space/4kdata 2020-09-12T14:27:32.222 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:32.222 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj50 td/osd-backfill-space/4kdata 2020-09-12T14:27:32.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:32.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj51 td/osd-backfill-space/4kdata 2020-09-12T14:27:32.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:32.466 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj52 td/osd-backfill-space/4kdata 2020-09-12T14:27:32.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:32.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj53 td/osd-backfill-space/4kdata 2020-09-12T14:27:32.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:32.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj54 td/osd-backfill-space/4kdata 2020-09-12T14:27:32.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:32.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj55 td/osd-backfill-space/4kdata 2020-09-12T14:27:32.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:32.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj56 td/osd-backfill-space/4kdata 2020-09-12T14:27:33.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:33.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj57 td/osd-backfill-space/4kdata 2020-09-12T14:27:33.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:33.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj58 td/osd-backfill-space/4kdata 2020-09-12T14:27:33.354 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:33.354 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj59 td/osd-backfill-space/4kdata 2020-09-12T14:27:33.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:33.480 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj60 td/osd-backfill-space/4kdata 2020-09-12T14:27:33.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:33.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj61 td/osd-backfill-space/4kdata 2020-09-12T14:27:33.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:33.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj62 td/osd-backfill-space/4kdata 2020-09-12T14:27:33.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:33.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj63 td/osd-backfill-space/4kdata 2020-09-12T14:27:34.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:34.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj64 td/osd-backfill-space/4kdata 2020-09-12T14:27:34.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:34.151 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj65 td/osd-backfill-space/4kdata 2020-09-12T14:27:34.284 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:34.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj66 td/osd-backfill-space/4kdata 2020-09-12T14:27:34.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:34.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj67 td/osd-backfill-space/4kdata 2020-09-12T14:27:34.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:34.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj68 td/osd-backfill-space/4kdata 2020-09-12T14:27:34.678 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:34.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj69 td/osd-backfill-space/4kdata 2020-09-12T14:27:34.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:34.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj70 td/osd-backfill-space/4kdata 2020-09-12T14:27:34.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:34.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj71 td/osd-backfill-space/4kdata 2020-09-12T14:27:35.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:35.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj72 td/osd-backfill-space/4kdata 2020-09-12T14:27:35.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:35.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj73 td/osd-backfill-space/4kdata 2020-09-12T14:27:35.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:35.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj74 td/osd-backfill-space/4kdata 2020-09-12T14:27:35.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:35.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj75 td/osd-backfill-space/4kdata 2020-09-12T14:27:35.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:35.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj76 td/osd-backfill-space/4kdata 2020-09-12T14:27:35.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:35.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj77 td/osd-backfill-space/4kdata 2020-09-12T14:27:35.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:35.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj78 td/osd-backfill-space/4kdata 2020-09-12T14:27:36.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:36.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj79 td/osd-backfill-space/4kdata 2020-09-12T14:27:36.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:36.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj80 td/osd-backfill-space/4kdata 2020-09-12T14:27:36.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:36.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj81 td/osd-backfill-space/4kdata 2020-09-12T14:27:36.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:36.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj82 td/osd-backfill-space/4kdata 2020-09-12T14:27:36.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:36.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj83 td/osd-backfill-space/4kdata 2020-09-12T14:27:36.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:36.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj84 td/osd-backfill-space/4kdata 2020-09-12T14:27:36.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:36.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj85 td/osd-backfill-space/4kdata 2020-09-12T14:27:36.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:36.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj86 td/osd-backfill-space/4kdata 2020-09-12T14:27:37.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:37.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj87 td/osd-backfill-space/4kdata 2020-09-12T14:27:37.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:37.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj88 td/osd-backfill-space/4kdata 2020-09-12T14:27:37.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:37.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj89 td/osd-backfill-space/4kdata 2020-09-12T14:27:37.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:37.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj90 td/osd-backfill-space/4kdata 2020-09-12T14:27:37.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:37.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj91 td/osd-backfill-space/4kdata 2020-09-12T14:27:37.771 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:37.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj92 td/osd-backfill-space/4kdata 2020-09-12T14:27:37.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:37.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj93 td/osd-backfill-space/4kdata 2020-09-12T14:27:38.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:38.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj94 td/osd-backfill-space/4kdata 2020-09-12T14:27:38.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:38.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj95 td/osd-backfill-space/4kdata 2020-09-12T14:27:38.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:38.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj96 td/osd-backfill-space/4kdata 2020-09-12T14:27:38.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:38.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj97 td/osd-backfill-space/4kdata 2020-09-12T14:27:38.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:38.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj98 td/osd-backfill-space/4kdata 2020-09-12T14:27:38.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:38.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj99 td/osd-backfill-space/4kdata 2020-09-12T14:27:38.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:38.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj100 td/osd-backfill-space/4kdata 2020-09-12T14:27:39.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:39.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj101 td/osd-backfill-space/4kdata 2020-09-12T14:27:39.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:39.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj102 td/osd-backfill-space/4kdata 2020-09-12T14:27:39.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:39.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj103 td/osd-backfill-space/4kdata 2020-09-12T14:27:39.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:39.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj104 td/osd-backfill-space/4kdata 2020-09-12T14:27:39.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:39.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj105 td/osd-backfill-space/4kdata 2020-09-12T14:27:39.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:39.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj106 td/osd-backfill-space/4kdata 2020-09-12T14:27:39.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:39.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj107 td/osd-backfill-space/4kdata 2020-09-12T14:27:39.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:39.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj108 td/osd-backfill-space/4kdata 2020-09-12T14:27:40.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:40.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj109 td/osd-backfill-space/4kdata 2020-09-12T14:27:40.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:40.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj110 td/osd-backfill-space/4kdata 2020-09-12T14:27:40.353 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:40.353 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj111 td/osd-backfill-space/4kdata 2020-09-12T14:27:40.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:40.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj112 td/osd-backfill-space/4kdata 2020-09-12T14:27:40.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:40.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj113 td/osd-backfill-space/4kdata 2020-09-12T14:27:40.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:40.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj114 td/osd-backfill-space/4kdata 2020-09-12T14:27:40.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:40.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj115 td/osd-backfill-space/4kdata 2020-09-12T14:27:41.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:41.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj116 td/osd-backfill-space/4kdata 2020-09-12T14:27:41.165 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:41.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj117 td/osd-backfill-space/4kdata 2020-09-12T14:27:41.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:41.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj118 td/osd-backfill-space/4kdata 2020-09-12T14:27:41.434 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:41.434 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj119 td/osd-backfill-space/4kdata 2020-09-12T14:27:41.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:41.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj120 td/osd-backfill-space/4kdata 2020-09-12T14:27:41.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:41.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj121 td/osd-backfill-space/4kdata 2020-09-12T14:27:41.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:41.862 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj122 td/osd-backfill-space/4kdata 2020-09-12T14:27:42.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:42.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj123 td/osd-backfill-space/4kdata 2020-09-12T14:27:42.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:42.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj124 td/osd-backfill-space/4kdata 2020-09-12T14:27:42.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:42.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj125 td/osd-backfill-space/4kdata 2020-09-12T14:27:42.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:42.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj126 td/osd-backfill-space/4kdata 2020-09-12T14:27:42.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:42.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj127 td/osd-backfill-space/4kdata 2020-09-12T14:27:42.649 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:42.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj128 td/osd-backfill-space/4kdata 2020-09-12T14:27:42.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:42.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj129 td/osd-backfill-space/4kdata 2020-09-12T14:27:42.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:42.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj130 td/osd-backfill-space/4kdata 2020-09-12T14:27:43.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:43.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj131 td/osd-backfill-space/4kdata 2020-09-12T14:27:43.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:43.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj132 td/osd-backfill-space/4kdata 2020-09-12T14:27:43.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:43.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj133 td/osd-backfill-space/4kdata 2020-09-12T14:27:43.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:43.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj134 td/osd-backfill-space/4kdata 2020-09-12T14:27:43.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:43.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj135 td/osd-backfill-space/4kdata 2020-09-12T14:27:43.730 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:43.730 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj136 td/osd-backfill-space/4kdata 2020-09-12T14:27:43.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:43.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj137 td/osd-backfill-space/4kdata 2020-09-12T14:27:44.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:44.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj138 td/osd-backfill-space/4kdata 2020-09-12T14:27:44.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:44.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj139 td/osd-backfill-space/4kdata 2020-09-12T14:27:44.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:44.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj140 td/osd-backfill-space/4kdata 2020-09-12T14:27:44.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:44.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj141 td/osd-backfill-space/4kdata 2020-09-12T14:27:44.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:44.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj142 td/osd-backfill-space/4kdata 2020-09-12T14:27:44.677 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:44.677 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj143 td/osd-backfill-space/4kdata 2020-09-12T14:27:44.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:44.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj144 td/osd-backfill-space/4kdata 2020-09-12T14:27:44.928 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:44.929 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj145 td/osd-backfill-space/4kdata 2020-09-12T14:27:45.062 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:45.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj146 td/osd-backfill-space/4kdata 2020-09-12T14:27:45.196 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:45.197 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj147 td/osd-backfill-space/4kdata 2020-09-12T14:27:45.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:45.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj148 td/osd-backfill-space/4kdata 2020-09-12T14:27:45.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:45.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj149 td/osd-backfill-space/4kdata 2020-09-12T14:27:45.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:45.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj150 td/osd-backfill-space/4kdata 2020-09-12T14:27:45.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:45.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj151 td/osd-backfill-space/4kdata 2020-09-12T14:27:45.859 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:45.859 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj152 td/osd-backfill-space/4kdata 2020-09-12T14:27:46.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:46.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj153 td/osd-backfill-space/4kdata 2020-09-12T14:27:46.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:46.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj154 td/osd-backfill-space/4kdata 2020-09-12T14:27:46.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:46.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj155 td/osd-backfill-space/4kdata 2020-09-12T14:27:46.454 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:46.454 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj156 td/osd-backfill-space/4kdata 2020-09-12T14:27:46.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:46.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj157 td/osd-backfill-space/4kdata 2020-09-12T14:27:46.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:46.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj158 td/osd-backfill-space/4kdata 2020-09-12T14:27:46.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:46.848 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj159 td/osd-backfill-space/4kdata 2020-09-12T14:27:46.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:46.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj160 td/osd-backfill-space/4kdata 2020-09-12T14:27:47.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:47.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj161 td/osd-backfill-space/4kdata 2020-09-12T14:27:47.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:47.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj162 td/osd-backfill-space/4kdata 2020-09-12T14:27:47.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:47.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj163 td/osd-backfill-space/4kdata 2020-09-12T14:27:47.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:47.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj164 td/osd-backfill-space/4kdata 2020-09-12T14:27:47.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:47.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj165 td/osd-backfill-space/4kdata 2020-09-12T14:27:47.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:47.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj166 td/osd-backfill-space/4kdata 2020-09-12T14:27:47.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:47.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj167 td/osd-backfill-space/4kdata 2020-09-12T14:27:48.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:48.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj168 td/osd-backfill-space/4kdata 2020-09-12T14:27:48.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:48.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj169 td/osd-backfill-space/4kdata 2020-09-12T14:27:48.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:48.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj170 td/osd-backfill-space/4kdata 2020-09-12T14:27:48.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:48.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj171 td/osd-backfill-space/4kdata 2020-09-12T14:27:48.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:48.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj172 td/osd-backfill-space/4kdata 2020-09-12T14:27:48.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:48.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj173 td/osd-backfill-space/4kdata 2020-09-12T14:27:48.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:48.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj174 td/osd-backfill-space/4kdata 2020-09-12T14:27:49.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:49.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj175 td/osd-backfill-space/4kdata 2020-09-12T14:27:49.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:49.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj176 td/osd-backfill-space/4kdata 2020-09-12T14:27:49.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:49.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj177 td/osd-backfill-space/4kdata 2020-09-12T14:27:49.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:49.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj178 td/osd-backfill-space/4kdata 2020-09-12T14:27:49.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:49.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj179 td/osd-backfill-space/4kdata 2020-09-12T14:27:49.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:49.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj180 td/osd-backfill-space/4kdata 2020-09-12T14:27:49.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:49.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj181 td/osd-backfill-space/4kdata 2020-09-12T14:27:49.932 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:49.932 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj182 td/osd-backfill-space/4kdata 2020-09-12T14:27:50.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:50.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj183 td/osd-backfill-space/4kdata 2020-09-12T14:27:50.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:50.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj184 td/osd-backfill-space/4kdata 2020-09-12T14:27:50.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:50.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj185 td/osd-backfill-space/4kdata 2020-09-12T14:27:50.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:50.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj186 td/osd-backfill-space/4kdata 2020-09-12T14:27:50.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:50.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj187 td/osd-backfill-space/4kdata 2020-09-12T14:27:50.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:50.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj188 td/osd-backfill-space/4kdata 2020-09-12T14:27:50.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:50.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj189 td/osd-backfill-space/4kdata 2020-09-12T14:27:50.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:50.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj190 td/osd-backfill-space/4kdata 2020-09-12T14:27:51.355 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:51.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj191 td/osd-backfill-space/4kdata 2020-09-12T14:27:51.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:51.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj192 td/osd-backfill-space/4kdata 2020-09-12T14:27:51.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:51.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj193 td/osd-backfill-space/4kdata 2020-09-12T14:27:51.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:51.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj194 td/osd-backfill-space/4kdata 2020-09-12T14:27:51.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:51.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj195 td/osd-backfill-space/4kdata 2020-09-12T14:27:52.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:52.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj196 td/osd-backfill-space/4kdata 2020-09-12T14:27:52.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:52.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj197 td/osd-backfill-space/4kdata 2020-09-12T14:27:52.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:52.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj198 td/osd-backfill-space/4kdata 2020-09-12T14:27:52.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:52.454 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj199 td/osd-backfill-space/4kdata 2020-09-12T14:27:52.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:52.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj200 td/osd-backfill-space/4kdata 2020-09-12T14:27:52.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:52.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj201 td/osd-backfill-space/4kdata 2020-09-12T14:27:52.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:52.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj202 td/osd-backfill-space/4kdata 2020-09-12T14:27:52.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:52.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj203 td/osd-backfill-space/4kdata 2020-09-12T14:27:53.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:53.116 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj204 td/osd-backfill-space/4kdata 2020-09-12T14:27:53.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:53.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj205 td/osd-backfill-space/4kdata 2020-09-12T14:27:53.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:53.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj206 td/osd-backfill-space/4kdata 2020-09-12T14:27:53.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:53.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj207 td/osd-backfill-space/4kdata 2020-09-12T14:27:53.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:53.644 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj208 td/osd-backfill-space/4kdata 2020-09-12T14:27:53.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:53.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj209 td/osd-backfill-space/4kdata 2020-09-12T14:27:53.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:53.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj210 td/osd-backfill-space/4kdata 2020-09-12T14:27:54.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:54.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj211 td/osd-backfill-space/4kdata 2020-09-12T14:27:54.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:54.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj212 td/osd-backfill-space/4kdata 2020-09-12T14:27:54.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:54.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj213 td/osd-backfill-space/4kdata 2020-09-12T14:27:54.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:54.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj214 td/osd-backfill-space/4kdata 2020-09-12T14:27:54.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:54.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj215 td/osd-backfill-space/4kdata 2020-09-12T14:27:54.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:54.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj216 td/osd-backfill-space/4kdata 2020-09-12T14:27:54.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:54.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj217 td/osd-backfill-space/4kdata 2020-09-12T14:27:55.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:55.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj218 td/osd-backfill-space/4kdata 2020-09-12T14:27:55.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:55.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj219 td/osd-backfill-space/4kdata 2020-09-12T14:27:55.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:55.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj220 td/osd-backfill-space/4kdata 2020-09-12T14:27:55.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:55.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj221 td/osd-backfill-space/4kdata 2020-09-12T14:27:55.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:55.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj222 td/osd-backfill-space/4kdata 2020-09-12T14:27:55.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:55.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj223 td/osd-backfill-space/4kdata 2020-09-12T14:27:55.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:55.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj224 td/osd-backfill-space/4kdata 2020-09-12T14:27:55.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:55.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj225 td/osd-backfill-space/4kdata 2020-09-12T14:27:56.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:56.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj226 td/osd-backfill-space/4kdata 2020-09-12T14:27:56.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:56.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj227 td/osd-backfill-space/4kdata 2020-09-12T14:27:56.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:56.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj228 td/osd-backfill-space/4kdata 2020-09-12T14:27:56.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:56.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj229 td/osd-backfill-space/4kdata 2020-09-12T14:27:56.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:56.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj230 td/osd-backfill-space/4kdata 2020-09-12T14:27:56.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:56.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj231 td/osd-backfill-space/4kdata 2020-09-12T14:27:56.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:56.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj232 td/osd-backfill-space/4kdata 2020-09-12T14:27:57.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:57.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj233 td/osd-backfill-space/4kdata 2020-09-12T14:27:57.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:57.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj234 td/osd-backfill-space/4kdata 2020-09-12T14:27:57.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:57.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj235 td/osd-backfill-space/4kdata 2020-09-12T14:27:57.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:57.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj236 td/osd-backfill-space/4kdata 2020-09-12T14:27:57.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:57.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj237 td/osd-backfill-space/4kdata 2020-09-12T14:27:57.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:57.659 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj238 td/osd-backfill-space/4kdata 2020-09-12T14:27:57.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:57.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj239 td/osd-backfill-space/4kdata 2020-09-12T14:27:57.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:57.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj240 td/osd-backfill-space/4kdata 2020-09-12T14:27:58.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:58.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj241 td/osd-backfill-space/4kdata 2020-09-12T14:27:58.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:58.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj242 td/osd-backfill-space/4kdata 2020-09-12T14:27:58.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:58.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj243 td/osd-backfill-space/4kdata 2020-09-12T14:27:58.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:58.447 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj244 td/osd-backfill-space/4kdata 2020-09-12T14:27:58.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:58.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj245 td/osd-backfill-space/4kdata 2020-09-12T14:27:58.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:58.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj246 td/osd-backfill-space/4kdata 2020-09-12T14:27:58.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:58.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj247 td/osd-backfill-space/4kdata 2020-09-12T14:27:58.933 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:58.933 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj248 td/osd-backfill-space/4kdata 2020-09-12T14:27:59.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:59.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj249 td/osd-backfill-space/4kdata 2020-09-12T14:27:59.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:59.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj250 td/osd-backfill-space/4kdata 2020-09-12T14:27:59.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:59.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj251 td/osd-backfill-space/4kdata 2020-09-12T14:27:59.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:59.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj252 td/osd-backfill-space/4kdata 2020-09-12T14:27:59.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:59.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj253 td/osd-backfill-space/4kdata 2020-09-12T14:27:59.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:59.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj254 td/osd-backfill-space/4kdata 2020-09-12T14:27:59.871 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:27:59.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj255 td/osd-backfill-space/4kdata 2020-09-12T14:28:00.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:00.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj256 td/osd-backfill-space/4kdata 2020-09-12T14:28:00.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:00.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj257 td/osd-backfill-space/4kdata 2020-09-12T14:28:00.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:00.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj258 td/osd-backfill-space/4kdata 2020-09-12T14:28:00.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:00.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj259 td/osd-backfill-space/4kdata 2020-09-12T14:28:00.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:00.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj260 td/osd-backfill-space/4kdata 2020-09-12T14:28:00.668 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:00.668 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj261 td/osd-backfill-space/4kdata 2020-09-12T14:28:00.785 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:00.785 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj262 td/osd-backfill-space/4kdata 2020-09-12T14:28:00.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:00.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj263 td/osd-backfill-space/4kdata 2020-09-12T14:28:01.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:01.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj264 td/osd-backfill-space/4kdata 2020-09-12T14:28:01.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:01.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj265 td/osd-backfill-space/4kdata 2020-09-12T14:28:01.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:01.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj266 td/osd-backfill-space/4kdata 2020-09-12T14:28:01.422 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:01.422 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj267 td/osd-backfill-space/4kdata 2020-09-12T14:28:01.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:01.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj268 td/osd-backfill-space/4kdata 2020-09-12T14:28:01.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:01.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj269 td/osd-backfill-space/4kdata 2020-09-12T14:28:01.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:01.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj270 td/osd-backfill-space/4kdata 2020-09-12T14:28:02.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:02.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj271 td/osd-backfill-space/4kdata 2020-09-12T14:28:02.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:02.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj272 td/osd-backfill-space/4kdata 2020-09-12T14:28:02.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:02.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj273 td/osd-backfill-space/4kdata 2020-09-12T14:28:02.378 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:02.378 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj274 td/osd-backfill-space/4kdata 2020-09-12T14:28:02.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:02.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj275 td/osd-backfill-space/4kdata 2020-09-12T14:28:02.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:02.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj276 td/osd-backfill-space/4kdata 2020-09-12T14:28:02.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:02.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj277 td/osd-backfill-space/4kdata 2020-09-12T14:28:02.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:02.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj278 td/osd-backfill-space/4kdata 2020-09-12T14:28:03.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:03.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj279 td/osd-backfill-space/4kdata 2020-09-12T14:28:03.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:03.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj280 td/osd-backfill-space/4kdata 2020-09-12T14:28:03.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:03.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj281 td/osd-backfill-space/4kdata 2020-09-12T14:28:03.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:03.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj282 td/osd-backfill-space/4kdata 2020-09-12T14:28:03.551 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:03.552 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj283 td/osd-backfill-space/4kdata 2020-09-12T14:28:03.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:03.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj284 td/osd-backfill-space/4kdata 2020-09-12T14:28:03.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:03.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj285 td/osd-backfill-space/4kdata 2020-09-12T14:28:03.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:03.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj286 td/osd-backfill-space/4kdata 2020-09-12T14:28:04.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:04.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj287 td/osd-backfill-space/4kdata 2020-09-12T14:28:04.264 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:04.264 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj288 td/osd-backfill-space/4kdata 2020-09-12T14:28:04.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:04.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj289 td/osd-backfill-space/4kdata 2020-09-12T14:28:04.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:04.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj290 td/osd-backfill-space/4kdata 2020-09-12T14:28:04.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:04.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj291 td/osd-backfill-space/4kdata 2020-09-12T14:28:04.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:04.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj292 td/osd-backfill-space/4kdata 2020-09-12T14:28:04.934 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:04.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj293 td/osd-backfill-space/4kdata 2020-09-12T14:28:05.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:05.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj294 td/osd-backfill-space/4kdata 2020-09-12T14:28:05.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:05.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj295 td/osd-backfill-space/4kdata 2020-09-12T14:28:05.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:05.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj296 td/osd-backfill-space/4kdata 2020-09-12T14:28:05.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:05.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj297 td/osd-backfill-space/4kdata 2020-09-12T14:28:05.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:05.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj298 td/osd-backfill-space/4kdata 2020-09-12T14:28:05.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:05.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj299 td/osd-backfill-space/4kdata 2020-09-12T14:28:05.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:05.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj300 td/osd-backfill-space/4kdata 2020-09-12T14:28:06.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:06.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj301 td/osd-backfill-space/4kdata 2020-09-12T14:28:06.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:06.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj302 td/osd-backfill-space/4kdata 2020-09-12T14:28:06.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:06.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj303 td/osd-backfill-space/4kdata 2020-09-12T14:28:06.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:06.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj304 td/osd-backfill-space/4kdata 2020-09-12T14:28:06.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:06.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj305 td/osd-backfill-space/4kdata 2020-09-12T14:28:06.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:06.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj306 td/osd-backfill-space/4kdata 2020-09-12T14:28:06.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:06.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj307 td/osd-backfill-space/4kdata 2020-09-12T14:28:06.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:06.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj308 td/osd-backfill-space/4kdata 2020-09-12T14:28:07.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:07.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj309 td/osd-backfill-space/4kdata 2020-09-12T14:28:07.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:07.231 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj310 td/osd-backfill-space/4kdata 2020-09-12T14:28:07.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:07.365 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj311 td/osd-backfill-space/4kdata 2020-09-12T14:28:07.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:07.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj312 td/osd-backfill-space/4kdata 2020-09-12T14:28:07.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:07.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj313 td/osd-backfill-space/4kdata 2020-09-12T14:28:07.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:07.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj314 td/osd-backfill-space/4kdata 2020-09-12T14:28:07.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:07.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj315 td/osd-backfill-space/4kdata 2020-09-12T14:28:08.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:08.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj316 td/osd-backfill-space/4kdata 2020-09-12T14:28:08.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:08.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj317 td/osd-backfill-space/4kdata 2020-09-12T14:28:08.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:08.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj318 td/osd-backfill-space/4kdata 2020-09-12T14:28:08.431 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:08.431 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj319 td/osd-backfill-space/4kdata 2020-09-12T14:28:08.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:08.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj320 td/osd-backfill-space/4kdata 2020-09-12T14:28:08.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:08.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj321 td/osd-backfill-space/4kdata 2020-09-12T14:28:08.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:08.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj322 td/osd-backfill-space/4kdata 2020-09-12T14:28:08.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:08.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj323 td/osd-backfill-space/4kdata 2020-09-12T14:28:09.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:09.119 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj324 td/osd-backfill-space/4kdata 2020-09-12T14:28:09.261 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:09.261 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj325 td/osd-backfill-space/4kdata 2020-09-12T14:28:09.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:09.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj326 td/osd-backfill-space/4kdata 2020-09-12T14:28:09.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:09.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj327 td/osd-backfill-space/4kdata 2020-09-12T14:28:09.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:09.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj328 td/osd-backfill-space/4kdata 2020-09-12T14:28:09.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:09.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj329 td/osd-backfill-space/4kdata 2020-09-12T14:28:10.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:10.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj330 td/osd-backfill-space/4kdata 2020-09-12T14:28:10.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:10.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj331 td/osd-backfill-space/4kdata 2020-09-12T14:28:10.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:10.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj332 td/osd-backfill-space/4kdata 2020-09-12T14:28:10.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:10.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj333 td/osd-backfill-space/4kdata 2020-09-12T14:28:10.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:10.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj334 td/osd-backfill-space/4kdata 2020-09-12T14:28:10.669 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:10.669 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj335 td/osd-backfill-space/4kdata 2020-09-12T14:28:10.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:10.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj336 td/osd-backfill-space/4kdata 2020-09-12T14:28:10.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:10.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj337 td/osd-backfill-space/4kdata 2020-09-12T14:28:11.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:11.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj338 td/osd-backfill-space/4kdata 2020-09-12T14:28:11.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:11.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj339 td/osd-backfill-space/4kdata 2020-09-12T14:28:11.347 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:11.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj340 td/osd-backfill-space/4kdata 2020-09-12T14:28:11.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:11.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj341 td/osd-backfill-space/4kdata 2020-09-12T14:28:11.590 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:11.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj342 td/osd-backfill-space/4kdata 2020-09-12T14:28:11.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:11.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj343 td/osd-backfill-space/4kdata 2020-09-12T14:28:11.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:11.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj344 td/osd-backfill-space/4kdata 2020-09-12T14:28:12.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:12.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj345 td/osd-backfill-space/4kdata 2020-09-12T14:28:12.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:12.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj346 td/osd-backfill-space/4kdata 2020-09-12T14:28:12.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:12.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj347 td/osd-backfill-space/4kdata 2020-09-12T14:28:12.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:12.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj348 td/osd-backfill-space/4kdata 2020-09-12T14:28:12.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:12.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj349 td/osd-backfill-space/4kdata 2020-09-12T14:28:12.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:12.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj350 td/osd-backfill-space/4kdata 2020-09-12T14:28:12.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:12.823 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj351 td/osd-backfill-space/4kdata 2020-09-12T14:28:12.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:12.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj352 td/osd-backfill-space/4kdata 2020-09-12T14:28:13.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:13.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj353 td/osd-backfill-space/4kdata 2020-09-12T14:28:13.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:13.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj354 td/osd-backfill-space/4kdata 2020-09-12T14:28:13.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:13.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj355 td/osd-backfill-space/4kdata 2020-09-12T14:28:13.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:13.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj356 td/osd-backfill-space/4kdata 2020-09-12T14:28:13.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:13.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj357 td/osd-backfill-space/4kdata 2020-09-12T14:28:13.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:13.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj358 td/osd-backfill-space/4kdata 2020-09-12T14:28:13.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:13.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj359 td/osd-backfill-space/4kdata 2020-09-12T14:28:13.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:13.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj360 td/osd-backfill-space/4kdata 2020-09-12T14:28:14.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:14.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj361 td/osd-backfill-space/4kdata 2020-09-12T14:28:14.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:14.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj362 td/osd-backfill-space/4kdata 2020-09-12T14:28:14.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:14.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj363 td/osd-backfill-space/4kdata 2020-09-12T14:28:14.433 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:14.433 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj364 td/osd-backfill-space/4kdata 2020-09-12T14:28:14.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:14.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj365 td/osd-backfill-space/4kdata 2020-09-12T14:28:14.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:14.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj366 td/osd-backfill-space/4kdata 2020-09-12T14:28:14.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:14.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj367 td/osd-backfill-space/4kdata 2020-09-12T14:28:14.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:14.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj368 td/osd-backfill-space/4kdata 2020-09-12T14:28:15.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:15.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj369 td/osd-backfill-space/4kdata 2020-09-12T14:28:15.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:15.222 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj370 td/osd-backfill-space/4kdata 2020-09-12T14:28:15.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:15.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj371 td/osd-backfill-space/4kdata 2020-09-12T14:28:15.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:15.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj372 td/osd-backfill-space/4kdata 2020-09-12T14:28:15.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:15.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj373 td/osd-backfill-space/4kdata 2020-09-12T14:28:15.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:15.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj374 td/osd-backfill-space/4kdata 2020-09-12T14:28:15.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:15.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj375 td/osd-backfill-space/4kdata 2020-09-12T14:28:15.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:15.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj376 td/osd-backfill-space/4kdata 2020-09-12T14:28:16.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:16.122 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj377 td/osd-backfill-space/4kdata 2020-09-12T14:28:16.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:16.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj378 td/osd-backfill-space/4kdata 2020-09-12T14:28:16.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:16.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj379 td/osd-backfill-space/4kdata 2020-09-12T14:28:16.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:16.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj380 td/osd-backfill-space/4kdata 2020-09-12T14:28:16.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:16.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj381 td/osd-backfill-space/4kdata 2020-09-12T14:28:16.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:16.765 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj382 td/osd-backfill-space/4kdata 2020-09-12T14:28:16.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:16.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj383 td/osd-backfill-space/4kdata 2020-09-12T14:28:17.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:17.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj384 td/osd-backfill-space/4kdata 2020-09-12T14:28:17.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:17.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj385 td/osd-backfill-space/4kdata 2020-09-12T14:28:17.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:17.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj386 td/osd-backfill-space/4kdata 2020-09-12T14:28:17.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:17.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj387 td/osd-backfill-space/4kdata 2020-09-12T14:28:17.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:17.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj388 td/osd-backfill-space/4kdata 2020-09-12T14:28:17.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:17.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj389 td/osd-backfill-space/4kdata 2020-09-12T14:28:17.862 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:17.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj390 td/osd-backfill-space/4kdata 2020-09-12T14:28:18.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:18.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj391 td/osd-backfill-space/4kdata 2020-09-12T14:28:18.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:18.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj392 td/osd-backfill-space/4kdata 2020-09-12T14:28:18.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:18.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj393 td/osd-backfill-space/4kdata 2020-09-12T14:28:18.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:18.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj394 td/osd-backfill-space/4kdata 2020-09-12T14:28:18.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:18.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj395 td/osd-backfill-space/4kdata 2020-09-12T14:28:18.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:18.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj396 td/osd-backfill-space/4kdata 2020-09-12T14:28:18.785 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:18.785 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj397 td/osd-backfill-space/4kdata 2020-09-12T14:28:18.944 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:18.944 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj398 td/osd-backfill-space/4kdata 2020-09-12T14:28:19.086 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:19.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj399 td/osd-backfill-space/4kdata 2020-09-12T14:28:19.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:19.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj400 td/osd-backfill-space/4kdata 2020-09-12T14:28:19.363 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:19.363 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj401 td/osd-backfill-space/4kdata 2020-09-12T14:28:19.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:19.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj402 td/osd-backfill-space/4kdata 2020-09-12T14:28:19.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:19.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj403 td/osd-backfill-space/4kdata 2020-09-12T14:28:19.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:19.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj404 td/osd-backfill-space/4kdata 2020-09-12T14:28:19.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:19.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj405 td/osd-backfill-space/4kdata 2020-09-12T14:28:20.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:20.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj406 td/osd-backfill-space/4kdata 2020-09-12T14:28:20.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:20.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj407 td/osd-backfill-space/4kdata 2020-09-12T14:28:20.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:20.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj408 td/osd-backfill-space/4kdata 2020-09-12T14:28:20.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:20.470 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj409 td/osd-backfill-space/4kdata 2020-09-12T14:28:20.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:20.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj410 td/osd-backfill-space/4kdata 2020-09-12T14:28:20.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:20.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj411 td/osd-backfill-space/4kdata 2020-09-12T14:28:20.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:20.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj412 td/osd-backfill-space/4kdata 2020-09-12T14:28:21.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:21.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj413 td/osd-backfill-space/4kdata 2020-09-12T14:28:21.165 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:21.165 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj414 td/osd-backfill-space/4kdata 2020-09-12T14:28:21.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:21.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj415 td/osd-backfill-space/4kdata 2020-09-12T14:28:21.433 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:21.434 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj416 td/osd-backfill-space/4kdata 2020-09-12T14:28:21.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:21.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj417 td/osd-backfill-space/4kdata 2020-09-12T14:28:21.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:21.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj418 td/osd-backfill-space/4kdata 2020-09-12T14:28:21.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:21.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj419 td/osd-backfill-space/4kdata 2020-09-12T14:28:22.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:22.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj420 td/osd-backfill-space/4kdata 2020-09-12T14:28:22.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:22.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj421 td/osd-backfill-space/4kdata 2020-09-12T14:28:22.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:22.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj422 td/osd-backfill-space/4kdata 2020-09-12T14:28:22.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:22.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj423 td/osd-backfill-space/4kdata 2020-09-12T14:28:22.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:22.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj424 td/osd-backfill-space/4kdata 2020-09-12T14:28:22.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:22.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj425 td/osd-backfill-space/4kdata 2020-09-12T14:28:23.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:23.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj426 td/osd-backfill-space/4kdata 2020-09-12T14:28:23.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:23.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj427 td/osd-backfill-space/4kdata 2020-09-12T14:28:23.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:23.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj428 td/osd-backfill-space/4kdata 2020-09-12T14:28:23.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:23.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj429 td/osd-backfill-space/4kdata 2020-09-12T14:28:23.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:23.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj430 td/osd-backfill-space/4kdata 2020-09-12T14:28:23.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:23.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj431 td/osd-backfill-space/4kdata 2020-09-12T14:28:23.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:23.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj432 td/osd-backfill-space/4kdata 2020-09-12T14:28:24.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:24.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj433 td/osd-backfill-space/4kdata 2020-09-12T14:28:24.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:24.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj434 td/osd-backfill-space/4kdata 2020-09-12T14:28:24.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:24.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj435 td/osd-backfill-space/4kdata 2020-09-12T14:28:24.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:24.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj436 td/osd-backfill-space/4kdata 2020-09-12T14:28:24.617 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:24.617 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj437 td/osd-backfill-space/4kdata 2020-09-12T14:28:24.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:24.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj438 td/osd-backfill-space/4kdata 2020-09-12T14:28:24.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:24.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj439 td/osd-backfill-space/4kdata 2020-09-12T14:28:25.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:25.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj440 td/osd-backfill-space/4kdata 2020-09-12T14:28:25.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:25.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj441 td/osd-backfill-space/4kdata 2020-09-12T14:28:25.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:25.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj442 td/osd-backfill-space/4kdata 2020-09-12T14:28:25.438 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:25.438 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj443 td/osd-backfill-space/4kdata 2020-09-12T14:28:25.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:25.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj444 td/osd-backfill-space/4kdata 2020-09-12T14:28:25.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:25.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj445 td/osd-backfill-space/4kdata 2020-09-12T14:28:25.841 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:25.841 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj446 td/osd-backfill-space/4kdata 2020-09-12T14:28:25.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:25.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj447 td/osd-backfill-space/4kdata 2020-09-12T14:28:26.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:26.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj448 td/osd-backfill-space/4kdata 2020-09-12T14:28:26.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:26.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj449 td/osd-backfill-space/4kdata 2020-09-12T14:28:26.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:26.378 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj450 td/osd-backfill-space/4kdata 2020-09-12T14:28:26.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:26.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj451 td/osd-backfill-space/4kdata 2020-09-12T14:28:26.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:26.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj452 td/osd-backfill-space/4kdata 2020-09-12T14:28:26.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:26.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj453 td/osd-backfill-space/4kdata 2020-09-12T14:28:26.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:26.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj454 td/osd-backfill-space/4kdata 2020-09-12T14:28:27.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:27.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj455 td/osd-backfill-space/4kdata 2020-09-12T14:28:27.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:27.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj456 td/osd-backfill-space/4kdata 2020-09-12T14:28:27.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:27.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj457 td/osd-backfill-space/4kdata 2020-09-12T14:28:27.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:27.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj458 td/osd-backfill-space/4kdata 2020-09-12T14:28:27.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:27.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj459 td/osd-backfill-space/4kdata 2020-09-12T14:28:27.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:27.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj460 td/osd-backfill-space/4kdata 2020-09-12T14:28:27.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:27.887 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj461 td/osd-backfill-space/4kdata 2020-09-12T14:28:28.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:28.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj462 td/osd-backfill-space/4kdata 2020-09-12T14:28:28.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:28.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj463 td/osd-backfill-space/4kdata 2020-09-12T14:28:28.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:28.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj464 td/osd-backfill-space/4kdata 2020-09-12T14:28:28.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:28.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj465 td/osd-backfill-space/4kdata 2020-09-12T14:28:28.573 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:28.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj466 td/osd-backfill-space/4kdata 2020-09-12T14:28:28.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:28.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj467 td/osd-backfill-space/4kdata 2020-09-12T14:28:28.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:28.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj468 td/osd-backfill-space/4kdata 2020-09-12T14:28:28.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:28.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj469 td/osd-backfill-space/4kdata 2020-09-12T14:28:29.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:29.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj470 td/osd-backfill-space/4kdata 2020-09-12T14:28:29.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:29.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj471 td/osd-backfill-space/4kdata 2020-09-12T14:28:29.353 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:29.353 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj472 td/osd-backfill-space/4kdata 2020-09-12T14:28:29.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:29.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj473 td/osd-backfill-space/4kdata 2020-09-12T14:28:29.621 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:29.621 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj474 td/osd-backfill-space/4kdata 2020-09-12T14:28:29.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:29.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj475 td/osd-backfill-space/4kdata 2020-09-12T14:28:29.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:29.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj476 td/osd-backfill-space/4kdata 2020-09-12T14:28:30.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:30.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj477 td/osd-backfill-space/4kdata 2020-09-12T14:28:30.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:30.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj478 td/osd-backfill-space/4kdata 2020-09-12T14:28:30.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:30.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj479 td/osd-backfill-space/4kdata 2020-09-12T14:28:30.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:30.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj480 td/osd-backfill-space/4kdata 2020-09-12T14:28:30.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:30.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj481 td/osd-backfill-space/4kdata 2020-09-12T14:28:30.728 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:30.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj482 td/osd-backfill-space/4kdata 2020-09-12T14:28:30.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:30.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj483 td/osd-backfill-space/4kdata 2020-09-12T14:28:31.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:31.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj484 td/osd-backfill-space/4kdata 2020-09-12T14:28:31.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:31.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj485 td/osd-backfill-space/4kdata 2020-09-12T14:28:31.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:31.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj486 td/osd-backfill-space/4kdata 2020-09-12T14:28:31.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:31.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj487 td/osd-backfill-space/4kdata 2020-09-12T14:28:31.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:31.550 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj488 td/osd-backfill-space/4kdata 2020-09-12T14:28:31.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:31.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj489 td/osd-backfill-space/4kdata 2020-09-12T14:28:31.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:31.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj490 td/osd-backfill-space/4kdata 2020-09-12T14:28:31.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:31.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj491 td/osd-backfill-space/4kdata 2020-09-12T14:28:32.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:32.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj492 td/osd-backfill-space/4kdata 2020-09-12T14:28:32.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:32.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj493 td/osd-backfill-space/4kdata 2020-09-12T14:28:32.371 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:32.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj494 td/osd-backfill-space/4kdata 2020-09-12T14:28:32.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:32.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj495 td/osd-backfill-space/4kdata 2020-09-12T14:28:32.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:32.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj496 td/osd-backfill-space/4kdata 2020-09-12T14:28:32.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:32.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj497 td/osd-backfill-space/4kdata 2020-09-12T14:28:32.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:32.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj498 td/osd-backfill-space/4kdata 2020-09-12T14:28:33.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:33.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj499 td/osd-backfill-space/4kdata 2020-09-12T14:28:33.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:33.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj500 td/osd-backfill-space/4kdata 2020-09-12T14:28:33.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:33.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj501 td/osd-backfill-space/4kdata 2020-09-12T14:28:33.482 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:33.482 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj502 td/osd-backfill-space/4kdata 2020-09-12T14:28:33.616 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:33.616 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj503 td/osd-backfill-space/4kdata 2020-09-12T14:28:33.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:33.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj504 td/osd-backfill-space/4kdata 2020-09-12T14:28:33.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:33.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj505 td/osd-backfill-space/4kdata 2020-09-12T14:28:34.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:34.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj506 td/osd-backfill-space/4kdata 2020-09-12T14:28:34.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:34.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj507 td/osd-backfill-space/4kdata 2020-09-12T14:28:34.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:34.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj508 td/osd-backfill-space/4kdata 2020-09-12T14:28:34.480 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:34.480 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj509 td/osd-backfill-space/4kdata 2020-09-12T14:28:34.606 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:34.606 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj510 td/osd-backfill-space/4kdata 2020-09-12T14:28:34.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:34.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj511 td/osd-backfill-space/4kdata 2020-09-12T14:28:34.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:34.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj512 td/osd-backfill-space/4kdata 2020-09-12T14:28:35.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:35.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj513 td/osd-backfill-space/4kdata 2020-09-12T14:28:35.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:35.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj514 td/osd-backfill-space/4kdata 2020-09-12T14:28:35.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:35.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj515 td/osd-backfill-space/4kdata 2020-09-12T14:28:35.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:35.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj516 td/osd-backfill-space/4kdata 2020-09-12T14:28:35.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:35.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj517 td/osd-backfill-space/4kdata 2020-09-12T14:28:35.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:35.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj518 td/osd-backfill-space/4kdata 2020-09-12T14:28:35.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:35.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj519 td/osd-backfill-space/4kdata 2020-09-12T14:28:35.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:35.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj520 td/osd-backfill-space/4kdata 2020-09-12T14:28:36.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:36.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj521 td/osd-backfill-space/4kdata 2020-09-12T14:28:36.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:36.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj522 td/osd-backfill-space/4kdata 2020-09-12T14:28:36.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:36.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj523 td/osd-backfill-space/4kdata 2020-09-12T14:28:36.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:36.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj524 td/osd-backfill-space/4kdata 2020-09-12T14:28:36.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:36.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj525 td/osd-backfill-space/4kdata 2020-09-12T14:28:36.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:36.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj526 td/osd-backfill-space/4kdata 2020-09-12T14:28:36.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:36.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj527 td/osd-backfill-space/4kdata 2020-09-12T14:28:37.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:37.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj528 td/osd-backfill-space/4kdata 2020-09-12T14:28:37.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:37.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj529 td/osd-backfill-space/4kdata 2020-09-12T14:28:37.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:37.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj530 td/osd-backfill-space/4kdata 2020-09-12T14:28:37.447 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:37.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj531 td/osd-backfill-space/4kdata 2020-09-12T14:28:37.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:37.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj532 td/osd-backfill-space/4kdata 2020-09-12T14:28:37.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:37.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj533 td/osd-backfill-space/4kdata 2020-09-12T14:28:37.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:37.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj534 td/osd-backfill-space/4kdata 2020-09-12T14:28:38.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:38.035 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj535 td/osd-backfill-space/4kdata 2020-09-12T14:28:38.177 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:38.177 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj536 td/osd-backfill-space/4kdata 2020-09-12T14:28:38.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:38.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj537 td/osd-backfill-space/4kdata 2020-09-12T14:28:38.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:38.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj538 td/osd-backfill-space/4kdata 2020-09-12T14:28:38.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:38.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj539 td/osd-backfill-space/4kdata 2020-09-12T14:28:38.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:38.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj540 td/osd-backfill-space/4kdata 2020-09-12T14:28:38.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:38.848 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj541 td/osd-backfill-space/4kdata 2020-09-12T14:28:38.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:38.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj542 td/osd-backfill-space/4kdata 2020-09-12T14:28:39.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:39.116 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj543 td/osd-backfill-space/4kdata 2020-09-12T14:28:39.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:39.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj544 td/osd-backfill-space/4kdata 2020-09-12T14:28:39.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:39.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj545 td/osd-backfill-space/4kdata 2020-09-12T14:28:39.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:39.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj546 td/osd-backfill-space/4kdata 2020-09-12T14:28:39.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:39.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj547 td/osd-backfill-space/4kdata 2020-09-12T14:28:39.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:39.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj548 td/osd-backfill-space/4kdata 2020-09-12T14:28:39.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:39.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj549 td/osd-backfill-space/4kdata 2020-09-12T14:28:40.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:40.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj550 td/osd-backfill-space/4kdata 2020-09-12T14:28:40.222 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:40.222 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj551 td/osd-backfill-space/4kdata 2020-09-12T14:28:40.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:40.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj552 td/osd-backfill-space/4kdata 2020-09-12T14:28:40.481 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:40.482 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj553 td/osd-backfill-space/4kdata 2020-09-12T14:28:40.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:40.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj554 td/osd-backfill-space/4kdata 2020-09-12T14:28:40.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:40.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj555 td/osd-backfill-space/4kdata 2020-09-12T14:28:40.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:40.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj556 td/osd-backfill-space/4kdata 2020-09-12T14:28:41.009 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:41.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj557 td/osd-backfill-space/4kdata 2020-09-12T14:28:41.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:41.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj558 td/osd-backfill-space/4kdata 2020-09-12T14:28:41.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:41.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj559 td/osd-backfill-space/4kdata 2020-09-12T14:28:41.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:41.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj560 td/osd-backfill-space/4kdata 2020-09-12T14:28:41.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:41.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj561 td/osd-backfill-space/4kdata 2020-09-12T14:28:41.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:41.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj562 td/osd-backfill-space/4kdata 2020-09-12T14:28:41.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:41.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj563 td/osd-backfill-space/4kdata 2020-09-12T14:28:41.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:41.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj564 td/osd-backfill-space/4kdata 2020-09-12T14:28:42.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:42.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj565 td/osd-backfill-space/4kdata 2020-09-12T14:28:42.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:42.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj566 td/osd-backfill-space/4kdata 2020-09-12T14:28:42.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:42.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj567 td/osd-backfill-space/4kdata 2020-09-12T14:28:42.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:42.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj568 td/osd-backfill-space/4kdata 2020-09-12T14:28:42.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:42.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj569 td/osd-backfill-space/4kdata 2020-09-12T14:28:42.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:42.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj570 td/osd-backfill-space/4kdata 2020-09-12T14:28:42.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:42.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj571 td/osd-backfill-space/4kdata 2020-09-12T14:28:42.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:42.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj572 td/osd-backfill-space/4kdata 2020-09-12T14:28:43.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:43.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj573 td/osd-backfill-space/4kdata 2020-09-12T14:28:43.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:43.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj574 td/osd-backfill-space/4kdata 2020-09-12T14:28:43.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:43.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj575 td/osd-backfill-space/4kdata 2020-09-12T14:28:43.466 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:43.466 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj576 td/osd-backfill-space/4kdata 2020-09-12T14:28:43.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:43.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj577 td/osd-backfill-space/4kdata 2020-09-12T14:28:43.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:43.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj578 td/osd-backfill-space/4kdata 2020-09-12T14:28:43.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:43.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj579 td/osd-backfill-space/4kdata 2020-09-12T14:28:43.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:43.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj580 td/osd-backfill-space/4kdata 2020-09-12T14:28:44.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:44.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj581 td/osd-backfill-space/4kdata 2020-09-12T14:28:44.246 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:44.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj582 td/osd-backfill-space/4kdata 2020-09-12T14:28:44.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:44.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj583 td/osd-backfill-space/4kdata 2020-09-12T14:28:44.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:44.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj584 td/osd-backfill-space/4kdata 2020-09-12T14:28:44.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:44.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj585 td/osd-backfill-space/4kdata 2020-09-12T14:28:44.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:44.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj586 td/osd-backfill-space/4kdata 2020-09-12T14:28:44.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:44.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj587 td/osd-backfill-space/4kdata 2020-09-12T14:28:45.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:45.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj588 td/osd-backfill-space/4kdata 2020-09-12T14:28:45.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:45.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj589 td/osd-backfill-space/4kdata 2020-09-12T14:28:45.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:45.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj590 td/osd-backfill-space/4kdata 2020-09-12T14:28:45.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:45.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj591 td/osd-backfill-space/4kdata 2020-09-12T14:28:45.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:45.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj592 td/osd-backfill-space/4kdata 2020-09-12T14:28:45.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:45.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj593 td/osd-backfill-space/4kdata 2020-09-12T14:28:45.841 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:45.841 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj594 td/osd-backfill-space/4kdata 2020-09-12T14:28:45.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:45.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj595 td/osd-backfill-space/4kdata 2020-09-12T14:28:46.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:46.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj596 td/osd-backfill-space/4kdata 2020-09-12T14:28:46.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:46.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj597 td/osd-backfill-space/4kdata 2020-09-12T14:28:46.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:46.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj598 td/osd-backfill-space/4kdata 2020-09-12T14:28:46.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:46.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj599 td/osd-backfill-space/4kdata 2020-09-12T14:28:46.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:578: TEST_backfill_grow: for i in $(seq 1 $objects) 2020-09-12T14:28:46.604 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:580: TEST_backfill_grow: rados -p test put obj600 td/osd-backfill-space/4kdata 2020-09-12T14:28:46.730 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:583: TEST_backfill_grow: get_pg test obj1 2020-09-12T14:28:46.731 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1031: get_pg: local poolname=test 2020-09-12T14:28:46.731 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1032: get_pg: local objectname=obj1 2020-09-12T14:28:46.731 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1034: get_pg: ceph --format json osd map test obj1 2020-09-12T14:28:46.731 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1034: get_pg: jq -r .pgid 2020-09-12T14:28:47.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:583: TEST_backfill_grow: local PG=1.0 2020-09-12T14:28:47.140 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:585: TEST_backfill_grow: get_primary test obj1 2020-09-12T14:28:47.140 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1134: get_primary: local poolname=test 2020-09-12T14:28:47.141 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1135: get_primary: local objectname=obj1 2020-09-12T14:28:47.141 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1137: get_primary: ceph --format json osd map test obj1 2020-09-12T14:28:47.141 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1138: get_primary: jq .acting_primary 2020-09-12T14:28:47.550 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:585: TEST_backfill_grow: local primary=1 2020-09-12T14:28:47.551 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:586: TEST_backfill_grow: get_not_primary test obj1 2020-09-12T14:28:47.551 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1167: get_not_primary: local poolname=test 2020-09-12T14:28:47.552 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: get_not_primary: local objectname=obj1 2020-09-12T14:28:47.552 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1170: get_not_primary: get_primary test obj1 2020-09-12T14:28:47.552 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1134: get_primary: local poolname=test 2020-09-12T14:28:47.552 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1135: get_primary: local objectname=obj1 2020-09-12T14:28:47.552 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1137: get_primary: ceph --format json osd map test obj1 2020-09-12T14:28:47.553 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1138: get_primary: jq .acting_primary 2020-09-12T14:28:47.958 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1170: get_not_primary: local primary=1 2020-09-12T14:28:47.959 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1171: get_not_primary: ceph --format json osd map test obj1 2020-09-12T14:28:47.959 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1172: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2020-09-12T14:28:48.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:586: TEST_backfill_grow: local otherosd=0 2020-09-12T14:28:48.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:588: TEST_backfill_grow: ceph osd set noout 2020-09-12T14:28:49.051 INFO:tasks.workunit.client.0.smithi027.stderr:noout is set 2020-09-12T14:28:49.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:589: TEST_backfill_grow: kill_daemons td/osd-backfill-space TERM 0 2020-09-12T14:28:49.062 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:28:49.062 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:28:49.062 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:28:49.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:28:49.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:28:49.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:28:49.169 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:591: TEST_backfill_grow: expr 600 / 4 2020-09-12T14:28:49.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:591: TEST_backfill_grow: rmobjects=150 2020-09-12T14:28:49.170 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: seq 1 150 2020-09-12T14:28:49.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:49.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj1 2020-09-12T14:28:49.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:49.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj2 2020-09-12T14:28:50.035 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:50.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj3 2020-09-12T14:28:50.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:50.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj4 2020-09-12T14:28:50.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:50.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj5 2020-09-12T14:28:50.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:50.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj6 2020-09-12T14:28:50.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:50.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj7 2020-09-12T14:28:50.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:50.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj8 2020-09-12T14:28:50.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:50.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj9 2020-09-12T14:28:50.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:50.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj10 2020-09-12T14:28:50.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:50.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj11 2020-09-12T14:28:50.943 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:50.943 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj12 2020-09-12T14:28:51.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:51.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj13 2020-09-12T14:28:51.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:51.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj14 2020-09-12T14:28:51.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:51.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj15 2020-09-12T14:28:51.353 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:51.354 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj16 2020-09-12T14:28:51.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:51.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj17 2020-09-12T14:28:51.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:51.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj18 2020-09-12T14:28:51.621 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:51.621 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj19 2020-09-12T14:28:51.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:51.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj20 2020-09-12T14:28:51.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:51.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj21 2020-09-12T14:28:51.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:51.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj22 2020-09-12T14:28:52.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:52.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj23 2020-09-12T14:28:52.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:52.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj24 2020-09-12T14:28:52.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:52.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj25 2020-09-12T14:28:52.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:52.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj26 2020-09-12T14:28:52.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:52.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj27 2020-09-12T14:28:52.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:52.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj28 2020-09-12T14:28:52.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:52.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj29 2020-09-12T14:28:52.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:52.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj30 2020-09-12T14:28:52.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:52.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj31 2020-09-12T14:28:52.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:52.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj32 2020-09-12T14:28:53.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:53.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj33 2020-09-12T14:28:53.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:53.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj34 2020-09-12T14:28:53.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:53.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj35 2020-09-12T14:28:53.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:53.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj36 2020-09-12T14:28:53.438 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:53.438 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj37 2020-09-12T14:28:53.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:53.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj38 2020-09-12T14:28:53.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:53.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj39 2020-09-12T14:28:54.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:54.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj40 2020-09-12T14:28:54.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:54.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj41 2020-09-12T14:28:54.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:54.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj42 2020-09-12T14:28:54.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:54.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj43 2020-09-12T14:28:54.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:54.417 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj44 2020-09-12T14:28:54.550 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:54.551 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj45 2020-09-12T14:28:54.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:54.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj46 2020-09-12T14:28:54.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:54.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj47 2020-09-12T14:28:54.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:54.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj48 2020-09-12T14:28:54.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:54.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj49 2020-09-12T14:28:54.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:54.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj50 2020-09-12T14:28:55.119 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:55.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj51 2020-09-12T14:28:55.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:55.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj52 2020-09-12T14:28:55.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:55.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj53 2020-09-12T14:28:55.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:55.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj54 2020-09-12T14:28:55.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:55.480 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj55 2020-09-12T14:28:55.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:55.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj56 2020-09-12T14:28:55.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:55.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj57 2020-09-12T14:28:55.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:55.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj58 2020-09-12T14:28:55.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:55.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj59 2020-09-12T14:28:56.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:56.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj60 2020-09-12T14:28:56.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:56.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj61 2020-09-12T14:28:56.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:56.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj62 2020-09-12T14:28:56.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:56.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj63 2020-09-12T14:28:56.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:56.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj64 2020-09-12T14:28:56.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:56.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj65 2020-09-12T14:28:56.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:56.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj66 2020-09-12T14:28:56.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:56.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj67 2020-09-12T14:28:56.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:56.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj68 2020-09-12T14:28:56.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:56.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj69 2020-09-12T14:28:56.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:56.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj70 2020-09-12T14:28:57.086 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:57.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj71 2020-09-12T14:28:57.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:57.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj72 2020-09-12T14:28:57.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:57.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj73 2020-09-12T14:28:57.371 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:57.371 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj74 2020-09-12T14:28:57.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:57.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj75 2020-09-12T14:28:57.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:57.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj76 2020-09-12T14:28:57.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:57.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj77 2020-09-12T14:28:57.765 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:57.765 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj78 2020-09-12T14:28:57.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:57.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj79 2020-09-12T14:28:58.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:58.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj80 2020-09-12T14:28:58.091 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:58.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj81 2020-09-12T14:28:58.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:58.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj82 2020-09-12T14:28:58.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:58.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj83 2020-09-12T14:28:58.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:58.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj84 2020-09-12T14:28:58.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:58.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj85 2020-09-12T14:28:58.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:58.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj86 2020-09-12T14:28:58.653 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:58.653 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj87 2020-09-12T14:28:58.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:58.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj88 2020-09-12T14:28:58.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:58.862 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj89 2020-09-12T14:28:58.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:58.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj90 2020-09-12T14:28:59.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:59.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj91 2020-09-12T14:28:59.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:59.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj92 2020-09-12T14:28:59.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:59.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj93 2020-09-12T14:28:59.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:59.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj94 2020-09-12T14:28:59.439 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:59.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj95 2020-09-12T14:28:59.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:59.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj96 2020-09-12T14:28:59.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:59.649 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj97 2020-09-12T14:28:59.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:59.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj98 2020-09-12T14:28:59.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:59.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj99 2020-09-12T14:28:59.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:28:59.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj100 2020-09-12T14:29:00.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:00.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj101 2020-09-12T14:29:00.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:00.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj102 2020-09-12T14:29:00.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:00.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj103 2020-09-12T14:29:00.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:00.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj104 2020-09-12T14:29:00.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:00.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj105 2020-09-12T14:29:00.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:00.612 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj106 2020-09-12T14:29:00.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:00.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj107 2020-09-12T14:29:00.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:00.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj108 2020-09-12T14:29:00.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:00.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj109 2020-09-12T14:29:00.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:00.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj110 2020-09-12T14:29:01.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:01.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj111 2020-09-12T14:29:01.197 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:01.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj112 2020-09-12T14:29:01.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:01.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj113 2020-09-12T14:29:01.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:01.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj114 2020-09-12T14:29:01.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:01.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj115 2020-09-12T14:29:01.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:01.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj116 2020-09-12T14:29:01.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:01.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj117 2020-09-12T14:29:01.892 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:01.892 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj118 2020-09-12T14:29:01.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:01.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj119 2020-09-12T14:29:02.085 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:02.085 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj120 2020-09-12T14:29:02.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:02.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj121 2020-09-12T14:29:02.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:02.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj122 2020-09-12T14:29:02.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:02.378 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj123 2020-09-12T14:29:02.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:02.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj124 2020-09-12T14:29:02.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:02.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj125 2020-09-12T14:29:02.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:02.671 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj126 2020-09-12T14:29:02.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:02.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj127 2020-09-12T14:29:02.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:02.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj128 2020-09-12T14:29:02.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:02.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj129 2020-09-12T14:29:03.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:03.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj130 2020-09-12T14:29:03.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:03.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj131 2020-09-12T14:29:03.298 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:03.298 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj132 2020-09-12T14:29:03.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:03.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj133 2020-09-12T14:29:03.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:03.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj134 2020-09-12T14:29:03.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:03.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj135 2020-09-12T14:29:03.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:03.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj136 2020-09-12T14:29:03.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:03.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj137 2020-09-12T14:29:03.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:03.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj138 2020-09-12T14:29:04.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:04.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj139 2020-09-12T14:29:04.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:04.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj140 2020-09-12T14:29:04.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:04.353 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj141 2020-09-12T14:29:04.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:04.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj142 2020-09-12T14:29:04.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:04.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj143 2020-09-12T14:29:04.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:04.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj144 2020-09-12T14:29:04.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:04.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj145 2020-09-12T14:29:04.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:04.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj146 2020-09-12T14:29:04.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:04.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj147 2020-09-12T14:29:05.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:05.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj148 2020-09-12T14:29:05.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:05.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj149 2020-09-12T14:29:05.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:592: TEST_backfill_grow: for i in $(seq 1 $rmobjects) 2020-09-12T14:29:05.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:594: TEST_backfill_grow: rados -p test rm obj150 2020-09-12T14:29:05.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:597: TEST_backfill_grow: dd if=/dev/urandom of=td/osd-backfill-space/6kdata bs=6k count=1 2020-09-12T14:29:05.341 INFO:tasks.workunit.client.0.smithi027.stderr:1+0 records in 2020-09-12T14:29:05.341 INFO:tasks.workunit.client.0.smithi027.stderr:1+0 records out 2020-09-12T14:29:05.341 INFO:tasks.workunit.client.0.smithi027.stderr:6144 bytes (6.1 kB, 6.0 KiB) copied, 7.5788e-05 s, 81.1 MB/s 2020-09-12T14:29:05.342 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: expr 150 + 1 2020-09-12T14:29:05.342 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: seq 151 600 2020-09-12T14:29:05.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:05.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj151 td/osd-backfill-space/6kdata 2020-09-12T14:29:05.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:05.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj152 td/osd-backfill-space/6kdata 2020-09-12T14:29:05.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:05.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj153 td/osd-backfill-space/6kdata 2020-09-12T14:29:05.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:05.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj154 td/osd-backfill-space/6kdata 2020-09-12T14:29:05.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:05.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj155 td/osd-backfill-space/6kdata 2020-09-12T14:29:05.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:05.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj156 td/osd-backfill-space/6kdata 2020-09-12T14:29:06.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:06.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj157 td/osd-backfill-space/6kdata 2020-09-12T14:29:06.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:06.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj158 td/osd-backfill-space/6kdata 2020-09-12T14:29:06.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:06.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj159 td/osd-backfill-space/6kdata 2020-09-12T14:29:06.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:06.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj160 td/osd-backfill-space/6kdata 2020-09-12T14:29:06.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:06.547 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj161 td/osd-backfill-space/6kdata 2020-09-12T14:29:06.672 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:06.673 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj162 td/osd-backfill-space/6kdata 2020-09-12T14:29:06.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:06.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj163 td/osd-backfill-space/6kdata 2020-09-12T14:29:06.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:06.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj164 td/osd-backfill-space/6kdata 2020-09-12T14:29:07.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:07.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj165 td/osd-backfill-space/6kdata 2020-09-12T14:29:07.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:07.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj166 td/osd-backfill-space/6kdata 2020-09-12T14:29:07.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:07.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj167 td/osd-backfill-space/6kdata 2020-09-12T14:29:07.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:07.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj168 td/osd-backfill-space/6kdata 2020-09-12T14:29:07.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:07.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj169 td/osd-backfill-space/6kdata 2020-09-12T14:29:07.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:07.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj170 td/osd-backfill-space/6kdata 2020-09-12T14:29:07.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:07.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj171 td/osd-backfill-space/6kdata 2020-09-12T14:29:07.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:07.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj172 td/osd-backfill-space/6kdata 2020-09-12T14:29:08.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:08.038 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj173 td/osd-backfill-space/6kdata 2020-09-12T14:29:08.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:08.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj174 td/osd-backfill-space/6kdata 2020-09-12T14:29:08.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:08.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj175 td/osd-backfill-space/6kdata 2020-09-12T14:29:08.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:08.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj176 td/osd-backfill-space/6kdata 2020-09-12T14:29:08.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:08.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj177 td/osd-backfill-space/6kdata 2020-09-12T14:29:08.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:08.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj178 td/osd-backfill-space/6kdata 2020-09-12T14:29:08.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:08.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj179 td/osd-backfill-space/6kdata 2020-09-12T14:29:08.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:08.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj180 td/osd-backfill-space/6kdata 2020-09-12T14:29:09.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:09.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj181 td/osd-backfill-space/6kdata 2020-09-12T14:29:09.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:09.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj182 td/osd-backfill-space/6kdata 2020-09-12T14:29:09.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:09.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj183 td/osd-backfill-space/6kdata 2020-09-12T14:29:09.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:09.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj184 td/osd-backfill-space/6kdata 2020-09-12T14:29:09.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:09.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj185 td/osd-backfill-space/6kdata 2020-09-12T14:29:09.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:09.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj186 td/osd-backfill-space/6kdata 2020-09-12T14:29:09.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:09.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj187 td/osd-backfill-space/6kdata 2020-09-12T14:29:09.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:09.892 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj188 td/osd-backfill-space/6kdata 2020-09-12T14:29:10.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:10.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj189 td/osd-backfill-space/6kdata 2020-09-12T14:29:10.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:10.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj190 td/osd-backfill-space/6kdata 2020-09-12T14:29:10.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:10.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj191 td/osd-backfill-space/6kdata 2020-09-12T14:29:10.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:10.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj192 td/osd-backfill-space/6kdata 2020-09-12T14:29:10.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:10.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj193 td/osd-backfill-space/6kdata 2020-09-12T14:29:10.667 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:10.667 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj194 td/osd-backfill-space/6kdata 2020-09-12T14:29:10.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:10.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj195 td/osd-backfill-space/6kdata 2020-09-12T14:29:10.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:10.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj196 td/osd-backfill-space/6kdata 2020-09-12T14:29:11.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:11.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj197 td/osd-backfill-space/6kdata 2020-09-12T14:29:11.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:11.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj198 td/osd-backfill-space/6kdata 2020-09-12T14:29:11.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:11.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj199 td/osd-backfill-space/6kdata 2020-09-12T14:29:11.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:11.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj200 td/osd-backfill-space/6kdata 2020-09-12T14:29:11.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:11.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj201 td/osd-backfill-space/6kdata 2020-09-12T14:29:11.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:11.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj202 td/osd-backfill-space/6kdata 2020-09-12T14:29:11.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:11.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj203 td/osd-backfill-space/6kdata 2020-09-12T14:29:11.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:11.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj204 td/osd-backfill-space/6kdata 2020-09-12T14:29:12.032 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:12.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj205 td/osd-backfill-space/6kdata 2020-09-12T14:29:12.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:12.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj206 td/osd-backfill-space/6kdata 2020-09-12T14:29:12.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:12.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj207 td/osd-backfill-space/6kdata 2020-09-12T14:29:12.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:12.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj208 td/osd-backfill-space/6kdata 2020-09-12T14:29:12.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:12.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj209 td/osd-backfill-space/6kdata 2020-09-12T14:29:12.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:12.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj210 td/osd-backfill-space/6kdata 2020-09-12T14:29:12.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:12.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj211 td/osd-backfill-space/6kdata 2020-09-12T14:29:12.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:12.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj212 td/osd-backfill-space/6kdata 2020-09-12T14:29:13.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:13.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj213 td/osd-backfill-space/6kdata 2020-09-12T14:29:13.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:13.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj214 td/osd-backfill-space/6kdata 2020-09-12T14:29:13.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:13.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj215 td/osd-backfill-space/6kdata 2020-09-12T14:29:13.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:13.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj216 td/osd-backfill-space/6kdata 2020-09-12T14:29:13.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:13.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj217 td/osd-backfill-space/6kdata 2020-09-12T14:29:13.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:13.649 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj218 td/osd-backfill-space/6kdata 2020-09-12T14:29:13.782 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:13.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj219 td/osd-backfill-space/6kdata 2020-09-12T14:29:13.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:13.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj220 td/osd-backfill-space/6kdata 2020-09-12T14:29:14.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:14.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj221 td/osd-backfill-space/6kdata 2020-09-12T14:29:14.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:14.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj222 td/osd-backfill-space/6kdata 2020-09-12T14:29:14.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:14.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj223 td/osd-backfill-space/6kdata 2020-09-12T14:29:14.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:14.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj224 td/osd-backfill-space/6kdata 2020-09-12T14:29:14.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:14.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj225 td/osd-backfill-space/6kdata 2020-09-12T14:29:14.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:14.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj226 td/osd-backfill-space/6kdata 2020-09-12T14:29:14.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:14.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj227 td/osd-backfill-space/6kdata 2020-09-12T14:29:14.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:14.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj228 td/osd-backfill-space/6kdata 2020-09-12T14:29:15.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:15.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj229 td/osd-backfill-space/6kdata 2020-09-12T14:29:15.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:15.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj230 td/osd-backfill-space/6kdata 2020-09-12T14:29:15.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:15.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj231 td/osd-backfill-space/6kdata 2020-09-12T14:29:15.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:15.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj232 td/osd-backfill-space/6kdata 2020-09-12T14:29:15.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:15.550 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj233 td/osd-backfill-space/6kdata 2020-09-12T14:29:15.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:15.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj234 td/osd-backfill-space/6kdata 2020-09-12T14:29:15.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:15.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj235 td/osd-backfill-space/6kdata 2020-09-12T14:29:15.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:15.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj236 td/osd-backfill-space/6kdata 2020-09-12T14:29:16.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:16.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj237 td/osd-backfill-space/6kdata 2020-09-12T14:29:16.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:16.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj238 td/osd-backfill-space/6kdata 2020-09-12T14:29:16.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:16.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj239 td/osd-backfill-space/6kdata 2020-09-12T14:29:16.454 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:16.454 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj240 td/osd-backfill-space/6kdata 2020-09-12T14:29:16.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:16.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj241 td/osd-backfill-space/6kdata 2020-09-12T14:29:16.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:16.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj242 td/osd-backfill-space/6kdata 2020-09-12T14:29:16.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:16.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj243 td/osd-backfill-space/6kdata 2020-09-12T14:29:16.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:16.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj244 td/osd-backfill-space/6kdata 2020-09-12T14:29:17.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:17.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj245 td/osd-backfill-space/6kdata 2020-09-12T14:29:17.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:17.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj246 td/osd-backfill-space/6kdata 2020-09-12T14:29:17.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:17.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj247 td/osd-backfill-space/6kdata 2020-09-12T14:29:17.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:17.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj248 td/osd-backfill-space/6kdata 2020-09-12T14:29:17.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:17.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj249 td/osd-backfill-space/6kdata 2020-09-12T14:29:17.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:17.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj250 td/osd-backfill-space/6kdata 2020-09-12T14:29:17.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:17.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj251 td/osd-backfill-space/6kdata 2020-09-12T14:29:17.944 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:17.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj252 td/osd-backfill-space/6kdata 2020-09-12T14:29:18.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:18.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj253 td/osd-backfill-space/6kdata 2020-09-12T14:29:18.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:18.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj254 td/osd-backfill-space/6kdata 2020-09-12T14:29:18.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:18.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj255 td/osd-backfill-space/6kdata 2020-09-12T14:29:18.438 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:18.439 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj256 td/osd-backfill-space/6kdata 2020-09-12T14:29:18.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:18.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj257 td/osd-backfill-space/6kdata 2020-09-12T14:29:18.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:18.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj258 td/osd-backfill-space/6kdata 2020-09-12T14:29:18.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:18.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj259 td/osd-backfill-space/6kdata 2020-09-12T14:29:18.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:18.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj260 td/osd-backfill-space/6kdata 2020-09-12T14:29:19.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:19.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj261 td/osd-backfill-space/6kdata 2020-09-12T14:29:19.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:19.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj262 td/osd-backfill-space/6kdata 2020-09-12T14:29:19.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:19.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj263 td/osd-backfill-space/6kdata 2020-09-12T14:29:19.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:19.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj264 td/osd-backfill-space/6kdata 2020-09-12T14:29:19.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:19.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj265 td/osd-backfill-space/6kdata 2020-09-12T14:29:19.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:19.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj266 td/osd-backfill-space/6kdata 2020-09-12T14:29:19.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:19.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj267 td/osd-backfill-space/6kdata 2020-09-12T14:29:19.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:19.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj268 td/osd-backfill-space/6kdata 2020-09-12T14:29:20.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:20.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj269 td/osd-backfill-space/6kdata 2020-09-12T14:29:20.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:20.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj270 td/osd-backfill-space/6kdata 2020-09-12T14:29:20.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:20.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj271 td/osd-backfill-space/6kdata 2020-09-12T14:29:20.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:20.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj272 td/osd-backfill-space/6kdata 2020-09-12T14:29:20.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:20.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj273 td/osd-backfill-space/6kdata 2020-09-12T14:29:20.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:20.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj274 td/osd-backfill-space/6kdata 2020-09-12T14:29:20.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:20.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj275 td/osd-backfill-space/6kdata 2020-09-12T14:29:20.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:20.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj276 td/osd-backfill-space/6kdata 2020-09-12T14:29:21.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:21.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj277 td/osd-backfill-space/6kdata 2020-09-12T14:29:21.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:21.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj278 td/osd-backfill-space/6kdata 2020-09-12T14:29:21.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:21.362 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj279 td/osd-backfill-space/6kdata 2020-09-12T14:29:21.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:21.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj280 td/osd-backfill-space/6kdata 2020-09-12T14:29:21.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:21.621 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj281 td/osd-backfill-space/6kdata 2020-09-12T14:29:21.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:21.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj282 td/osd-backfill-space/6kdata 2020-09-12T14:29:21.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:21.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj283 td/osd-backfill-space/6kdata 2020-09-12T14:29:22.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:22.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj284 td/osd-backfill-space/6kdata 2020-09-12T14:29:22.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:22.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj285 td/osd-backfill-space/6kdata 2020-09-12T14:29:22.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:22.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj286 td/osd-backfill-space/6kdata 2020-09-12T14:29:22.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:22.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj287 td/osd-backfill-space/6kdata 2020-09-12T14:29:22.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:22.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj288 td/osd-backfill-space/6kdata 2020-09-12T14:29:22.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:22.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj289 td/osd-backfill-space/6kdata 2020-09-12T14:29:22.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:22.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj290 td/osd-backfill-space/6kdata 2020-09-12T14:29:22.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:22.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj291 td/osd-backfill-space/6kdata 2020-09-12T14:29:23.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:23.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj292 td/osd-backfill-space/6kdata 2020-09-12T14:29:23.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:23.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj293 td/osd-backfill-space/6kdata 2020-09-12T14:29:23.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:23.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj294 td/osd-backfill-space/6kdata 2020-09-12T14:29:23.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:23.371 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj295 td/osd-backfill-space/6kdata 2020-09-12T14:29:23.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:23.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj296 td/osd-backfill-space/6kdata 2020-09-12T14:29:23.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:23.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj297 td/osd-backfill-space/6kdata 2020-09-12T14:29:23.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:23.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj298 td/osd-backfill-space/6kdata 2020-09-12T14:29:23.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:23.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj299 td/osd-backfill-space/6kdata 2020-09-12T14:29:23.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:23.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj300 td/osd-backfill-space/6kdata 2020-09-12T14:29:24.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:24.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj301 td/osd-backfill-space/6kdata 2020-09-12T14:29:24.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:24.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj302 td/osd-backfill-space/6kdata 2020-09-12T14:29:24.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:24.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj303 td/osd-backfill-space/6kdata 2020-09-12T14:29:24.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:24.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj304 td/osd-backfill-space/6kdata 2020-09-12T14:29:24.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:24.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj305 td/osd-backfill-space/6kdata 2020-09-12T14:29:24.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:24.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj306 td/osd-backfill-space/6kdata 2020-09-12T14:29:25.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:25.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj307 td/osd-backfill-space/6kdata 2020-09-12T14:29:25.246 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:25.246 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj308 td/osd-backfill-space/6kdata 2020-09-12T14:29:25.371 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:25.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj309 td/osd-backfill-space/6kdata 2020-09-12T14:29:25.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:25.498 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj310 td/osd-backfill-space/6kdata 2020-09-12T14:29:25.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:25.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj311 td/osd-backfill-space/6kdata 2020-09-12T14:29:25.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:25.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj312 td/osd-backfill-space/6kdata 2020-09-12T14:29:25.908 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:25.908 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj313 td/osd-backfill-space/6kdata 2020-09-12T14:29:26.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:26.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj314 td/osd-backfill-space/6kdata 2020-09-12T14:29:26.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:26.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj315 td/osd-backfill-space/6kdata 2020-09-12T14:29:26.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:26.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj316 td/osd-backfill-space/6kdata 2020-09-12T14:29:26.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:26.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj317 td/osd-backfill-space/6kdata 2020-09-12T14:29:26.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:26.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj318 td/osd-backfill-space/6kdata 2020-09-12T14:29:26.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:26.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj319 td/osd-backfill-space/6kdata 2020-09-12T14:29:26.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:26.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj320 td/osd-backfill-space/6kdata 2020-09-12T14:29:26.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:26.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj321 td/osd-backfill-space/6kdata 2020-09-12T14:29:27.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:27.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj322 td/osd-backfill-space/6kdata 2020-09-12T14:29:27.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:27.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj323 td/osd-backfill-space/6kdata 2020-09-12T14:29:27.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:27.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj324 td/osd-backfill-space/6kdata 2020-09-12T14:29:27.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:27.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj325 td/osd-backfill-space/6kdata 2020-09-12T14:29:27.590 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:27.590 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj326 td/osd-backfill-space/6kdata 2020-09-12T14:29:27.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:27.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj327 td/osd-backfill-space/6kdata 2020-09-12T14:29:27.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:27.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj328 td/osd-backfill-space/6kdata 2020-09-12T14:29:27.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:27.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj329 td/osd-backfill-space/6kdata 2020-09-12T14:29:28.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:28.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj330 td/osd-backfill-space/6kdata 2020-09-12T14:29:28.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:28.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj331 td/osd-backfill-space/6kdata 2020-09-12T14:29:28.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:28.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj332 td/osd-backfill-space/6kdata 2020-09-12T14:29:28.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:28.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj333 td/osd-backfill-space/6kdata 2020-09-12T14:29:28.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:28.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj334 td/osd-backfill-space/6kdata 2020-09-12T14:29:28.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:28.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj335 td/osd-backfill-space/6kdata 2020-09-12T14:29:28.871 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:28.871 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj336 td/osd-backfill-space/6kdata 2020-09-12T14:29:28.997 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:28.997 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj337 td/osd-backfill-space/6kdata 2020-09-12T14:29:29.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:29.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj338 td/osd-backfill-space/6kdata 2020-09-12T14:29:29.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:29.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj339 td/osd-backfill-space/6kdata 2020-09-12T14:29:29.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:29.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj340 td/osd-backfill-space/6kdata 2020-09-12T14:29:29.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:29.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj341 td/osd-backfill-space/6kdata 2020-09-12T14:29:29.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:29.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj342 td/osd-backfill-space/6kdata 2020-09-12T14:29:29.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:29.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj343 td/osd-backfill-space/6kdata 2020-09-12T14:29:29.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:29.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj344 td/osd-backfill-space/6kdata 2020-09-12T14:29:30.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:30.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj345 td/osd-backfill-space/6kdata 2020-09-12T14:29:30.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:30.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj346 td/osd-backfill-space/6kdata 2020-09-12T14:29:30.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:30.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj347 td/osd-backfill-space/6kdata 2020-09-12T14:29:30.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:30.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj348 td/osd-backfill-space/6kdata 2020-09-12T14:29:30.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:30.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj349 td/osd-backfill-space/6kdata 2020-09-12T14:29:30.671 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:30.672 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj350 td/osd-backfill-space/6kdata 2020-09-12T14:29:30.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:30.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj351 td/osd-backfill-space/6kdata 2020-09-12T14:29:30.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:30.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj352 td/osd-backfill-space/6kdata 2020-09-12T14:29:31.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:31.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj353 td/osd-backfill-space/6kdata 2020-09-12T14:29:31.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:31.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj354 td/osd-backfill-space/6kdata 2020-09-12T14:29:31.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:31.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj355 td/osd-backfill-space/6kdata 2020-09-12T14:29:31.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:31.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj356 td/osd-backfill-space/6kdata 2020-09-12T14:29:31.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:31.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj357 td/osd-backfill-space/6kdata 2020-09-12T14:29:31.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:31.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj358 td/osd-backfill-space/6kdata 2020-09-12T14:29:31.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:31.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj359 td/osd-backfill-space/6kdata 2020-09-12T14:29:31.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:31.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj360 td/osd-backfill-space/6kdata 2020-09-12T14:29:32.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:32.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj361 td/osd-backfill-space/6kdata 2020-09-12T14:29:32.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:32.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj362 td/osd-backfill-space/6kdata 2020-09-12T14:29:32.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:32.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj363 td/osd-backfill-space/6kdata 2020-09-12T14:29:32.447 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:32.447 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj364 td/osd-backfill-space/6kdata 2020-09-12T14:29:32.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:32.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj365 td/osd-backfill-space/6kdata 2020-09-12T14:29:32.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:32.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj366 td/osd-backfill-space/6kdata 2020-09-12T14:29:32.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:32.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj367 td/osd-backfill-space/6kdata 2020-09-12T14:29:32.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:32.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj368 td/osd-backfill-space/6kdata 2020-09-12T14:29:33.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:33.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj369 td/osd-backfill-space/6kdata 2020-09-12T14:29:33.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:33.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj370 td/osd-backfill-space/6kdata 2020-09-12T14:29:33.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:33.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj371 td/osd-backfill-space/6kdata 2020-09-12T14:29:33.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:33.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj372 td/osd-backfill-space/6kdata 2020-09-12T14:29:33.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:33.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj373 td/osd-backfill-space/6kdata 2020-09-12T14:29:33.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:33.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj374 td/osd-backfill-space/6kdata 2020-09-12T14:29:33.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:33.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj375 td/osd-backfill-space/6kdata 2020-09-12T14:29:33.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:33.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj376 td/osd-backfill-space/6kdata 2020-09-12T14:29:34.113 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:34.113 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj377 td/osd-backfill-space/6kdata 2020-09-12T14:29:34.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:34.240 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj378 td/osd-backfill-space/6kdata 2020-09-12T14:29:34.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:34.365 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj379 td/osd-backfill-space/6kdata 2020-09-12T14:29:34.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:34.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj380 td/osd-backfill-space/6kdata 2020-09-12T14:29:34.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:34.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj381 td/osd-backfill-space/6kdata 2020-09-12T14:29:34.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:34.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj382 td/osd-backfill-space/6kdata 2020-09-12T14:29:34.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:34.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj383 td/osd-backfill-space/6kdata 2020-09-12T14:29:34.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:34.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj384 td/osd-backfill-space/6kdata 2020-09-12T14:29:35.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:35.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj385 td/osd-backfill-space/6kdata 2020-09-12T14:29:35.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:35.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj386 td/osd-backfill-space/6kdata 2020-09-12T14:29:35.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:35.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj387 td/osd-backfill-space/6kdata 2020-09-12T14:29:35.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:35.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj388 td/osd-backfill-space/6kdata 2020-09-12T14:29:35.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:35.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj389 td/osd-backfill-space/6kdata 2020-09-12T14:29:35.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:35.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj390 td/osd-backfill-space/6kdata 2020-09-12T14:29:35.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:35.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj391 td/osd-backfill-space/6kdata 2020-09-12T14:29:36.014 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:36.014 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj392 td/osd-backfill-space/6kdata 2020-09-12T14:29:36.165 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:36.165 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj393 td/osd-backfill-space/6kdata 2020-09-12T14:29:36.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:36.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj394 td/osd-backfill-space/6kdata 2020-09-12T14:29:36.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:36.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj395 td/osd-backfill-space/6kdata 2020-09-12T14:29:36.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:36.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj396 td/osd-backfill-space/6kdata 2020-09-12T14:29:36.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:36.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj397 td/osd-backfill-space/6kdata 2020-09-12T14:29:36.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:36.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj398 td/osd-backfill-space/6kdata 2020-09-12T14:29:36.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:36.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj399 td/osd-backfill-space/6kdata 2020-09-12T14:29:37.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:37.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj400 td/osd-backfill-space/6kdata 2020-09-12T14:29:37.119 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:37.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj401 td/osd-backfill-space/6kdata 2020-09-12T14:29:37.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:37.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj402 td/osd-backfill-space/6kdata 2020-09-12T14:29:37.371 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:37.371 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj403 td/osd-backfill-space/6kdata 2020-09-12T14:29:37.496 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:37.496 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj404 td/osd-backfill-space/6kdata 2020-09-12T14:29:37.622 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:37.622 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj405 td/osd-backfill-space/6kdata 2020-09-12T14:29:37.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:37.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj406 td/osd-backfill-space/6kdata 2020-09-12T14:29:37.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:37.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj407 td/osd-backfill-space/6kdata 2020-09-12T14:29:38.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:38.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj408 td/osd-backfill-space/6kdata 2020-09-12T14:29:38.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:38.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj409 td/osd-backfill-space/6kdata 2020-09-12T14:29:38.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:38.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj410 td/osd-backfill-space/6kdata 2020-09-12T14:29:38.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:38.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj411 td/osd-backfill-space/6kdata 2020-09-12T14:29:38.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:38.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj412 td/osd-backfill-space/6kdata 2020-09-12T14:29:38.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:38.644 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj413 td/osd-backfill-space/6kdata 2020-09-12T14:29:38.761 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:38.761 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj414 td/osd-backfill-space/6kdata 2020-09-12T14:29:38.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:38.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj415 td/osd-backfill-space/6kdata 2020-09-12T14:29:38.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:38.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj416 td/osd-backfill-space/6kdata 2020-09-12T14:29:39.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:39.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj417 td/osd-backfill-space/6kdata 2020-09-12T14:29:39.246 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:39.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj418 td/osd-backfill-space/6kdata 2020-09-12T14:29:39.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:39.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj419 td/osd-backfill-space/6kdata 2020-09-12T14:29:39.481 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:39.481 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj420 td/osd-backfill-space/6kdata 2020-09-12T14:29:39.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:39.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj421 td/osd-backfill-space/6kdata 2020-09-12T14:29:39.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:39.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj422 td/osd-backfill-space/6kdata 2020-09-12T14:29:39.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:39.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj423 td/osd-backfill-space/6kdata 2020-09-12T14:29:40.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:40.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj424 td/osd-backfill-space/6kdata 2020-09-12T14:29:40.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:40.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj425 td/osd-backfill-space/6kdata 2020-09-12T14:29:40.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:40.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj426 td/osd-backfill-space/6kdata 2020-09-12T14:29:40.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:40.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj427 td/osd-backfill-space/6kdata 2020-09-12T14:29:40.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:40.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj428 td/osd-backfill-space/6kdata 2020-09-12T14:29:40.653 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:40.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj429 td/osd-backfill-space/6kdata 2020-09-12T14:29:40.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:40.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj430 td/osd-backfill-space/6kdata 2020-09-12T14:29:40.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:40.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj431 td/osd-backfill-space/6kdata 2020-09-12T14:29:41.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:41.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj432 td/osd-backfill-space/6kdata 2020-09-12T14:29:41.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:41.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj433 td/osd-backfill-space/6kdata 2020-09-12T14:29:41.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:41.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj434 td/osd-backfill-space/6kdata 2020-09-12T14:29:41.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:41.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj435 td/osd-backfill-space/6kdata 2020-09-12T14:29:41.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:41.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj436 td/osd-backfill-space/6kdata 2020-09-12T14:29:41.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:41.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj437 td/osd-backfill-space/6kdata 2020-09-12T14:29:41.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:41.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj438 td/osd-backfill-space/6kdata 2020-09-12T14:29:41.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:41.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj439 td/osd-backfill-space/6kdata 2020-09-12T14:29:42.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:42.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj440 td/osd-backfill-space/6kdata 2020-09-12T14:29:42.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:42.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj441 td/osd-backfill-space/6kdata 2020-09-12T14:29:42.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:42.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj442 td/osd-backfill-space/6kdata 2020-09-12T14:29:42.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:42.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj443 td/osd-backfill-space/6kdata 2020-09-12T14:29:42.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:42.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj444 td/osd-backfill-space/6kdata 2020-09-12T14:29:42.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:42.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj445 td/osd-backfill-space/6kdata 2020-09-12T14:29:42.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:42.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj446 td/osd-backfill-space/6kdata 2020-09-12T14:29:42.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:42.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj447 td/osd-backfill-space/6kdata 2020-09-12T14:29:43.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:43.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj448 td/osd-backfill-space/6kdata 2020-09-12T14:29:43.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:43.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj449 td/osd-backfill-space/6kdata 2020-09-12T14:29:43.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:43.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj450 td/osd-backfill-space/6kdata 2020-09-12T14:29:43.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:43.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj451 td/osd-backfill-space/6kdata 2020-09-12T14:29:43.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:43.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj452 td/osd-backfill-space/6kdata 2020-09-12T14:29:43.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:43.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj453 td/osd-backfill-space/6kdata 2020-09-12T14:29:43.871 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:43.871 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj454 td/osd-backfill-space/6kdata 2020-09-12T14:29:44.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:44.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj455 td/osd-backfill-space/6kdata 2020-09-12T14:29:44.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:44.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj456 td/osd-backfill-space/6kdata 2020-09-12T14:29:44.264 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:44.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj457 td/osd-backfill-space/6kdata 2020-09-12T14:29:44.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:44.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj458 td/osd-backfill-space/6kdata 2020-09-12T14:29:44.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:44.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj459 td/osd-backfill-space/6kdata 2020-09-12T14:29:44.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:44.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj460 td/osd-backfill-space/6kdata 2020-09-12T14:29:44.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:44.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj461 td/osd-backfill-space/6kdata 2020-09-12T14:29:44.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:44.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj462 td/osd-backfill-space/6kdata 2020-09-12T14:29:45.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:45.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj463 td/osd-backfill-space/6kdata 2020-09-12T14:29:45.177 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:45.177 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj464 td/osd-backfill-space/6kdata 2020-09-12T14:29:45.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:45.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj465 td/osd-backfill-space/6kdata 2020-09-12T14:29:45.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:45.454 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj466 td/osd-backfill-space/6kdata 2020-09-12T14:29:45.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:45.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj467 td/osd-backfill-space/6kdata 2020-09-12T14:29:45.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:45.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj468 td/osd-backfill-space/6kdata 2020-09-12T14:29:45.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:45.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj469 td/osd-backfill-space/6kdata 2020-09-12T14:29:46.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:46.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj470 td/osd-backfill-space/6kdata 2020-09-12T14:29:46.165 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:46.165 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj471 td/osd-backfill-space/6kdata 2020-09-12T14:29:46.291 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:46.291 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj472 td/osd-backfill-space/6kdata 2020-09-12T14:29:46.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:46.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj473 td/osd-backfill-space/6kdata 2020-09-12T14:29:46.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:46.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj474 td/osd-backfill-space/6kdata 2020-09-12T14:29:46.667 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:46.667 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj475 td/osd-backfill-space/6kdata 2020-09-12T14:29:46.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:46.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj476 td/osd-backfill-space/6kdata 2020-09-12T14:29:46.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:46.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj477 td/osd-backfill-space/6kdata 2020-09-12T14:29:47.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:47.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj478 td/osd-backfill-space/6kdata 2020-09-12T14:29:47.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:47.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj479 td/osd-backfill-space/6kdata 2020-09-12T14:29:47.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:47.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj480 td/osd-backfill-space/6kdata 2020-09-12T14:29:47.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:47.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj481 td/osd-backfill-space/6kdata 2020-09-12T14:29:47.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:47.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj482 td/osd-backfill-space/6kdata 2020-09-12T14:29:47.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:47.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj483 td/osd-backfill-space/6kdata 2020-09-12T14:29:47.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:47.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj484 td/osd-backfill-space/6kdata 2020-09-12T14:29:47.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:47.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj485 td/osd-backfill-space/6kdata 2020-09-12T14:29:48.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:48.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj486 td/osd-backfill-space/6kdata 2020-09-12T14:29:48.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:48.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj487 td/osd-backfill-space/6kdata 2020-09-12T14:29:48.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:48.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj488 td/osd-backfill-space/6kdata 2020-09-12T14:29:48.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:48.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj489 td/osd-backfill-space/6kdata 2020-09-12T14:29:48.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:48.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj490 td/osd-backfill-space/6kdata 2020-09-12T14:29:48.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:48.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj491 td/osd-backfill-space/6kdata 2020-09-12T14:29:48.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:48.761 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj492 td/osd-backfill-space/6kdata 2020-09-12T14:29:48.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:48.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj493 td/osd-backfill-space/6kdata 2020-09-12T14:29:49.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:49.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj494 td/osd-backfill-space/6kdata 2020-09-12T14:29:49.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:49.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj495 td/osd-backfill-space/6kdata 2020-09-12T14:29:49.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:49.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj496 td/osd-backfill-space/6kdata 2020-09-12T14:29:49.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:49.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj497 td/osd-backfill-space/6kdata 2020-09-12T14:29:49.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:49.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj498 td/osd-backfill-space/6kdata 2020-09-12T14:29:49.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:49.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj499 td/osd-backfill-space/6kdata 2020-09-12T14:29:49.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:49.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj500 td/osd-backfill-space/6kdata 2020-09-12T14:29:49.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:49.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj501 td/osd-backfill-space/6kdata 2020-09-12T14:29:50.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:50.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj502 td/osd-backfill-space/6kdata 2020-09-12T14:29:50.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:50.151 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj503 td/osd-backfill-space/6kdata 2020-09-12T14:29:50.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:50.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj504 td/osd-backfill-space/6kdata 2020-09-12T14:29:50.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:50.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj505 td/osd-backfill-space/6kdata 2020-09-12T14:29:50.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:50.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj506 td/osd-backfill-space/6kdata 2020-09-12T14:29:50.644 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:50.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj507 td/osd-backfill-space/6kdata 2020-09-12T14:29:50.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:50.771 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj508 td/osd-backfill-space/6kdata 2020-09-12T14:29:50.929 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:50.929 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj509 td/osd-backfill-space/6kdata 2020-09-12T14:29:51.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:51.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj510 td/osd-backfill-space/6kdata 2020-09-12T14:29:51.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:51.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj511 td/osd-backfill-space/6kdata 2020-09-12T14:29:51.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:51.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj512 td/osd-backfill-space/6kdata 2020-09-12T14:29:51.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:51.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj513 td/osd-backfill-space/6kdata 2020-09-12T14:29:51.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:51.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj514 td/osd-backfill-space/6kdata 2020-09-12T14:29:51.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:51.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj515 td/osd-backfill-space/6kdata 2020-09-12T14:29:51.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:51.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj516 td/osd-backfill-space/6kdata 2020-09-12T14:29:51.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:51.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj517 td/osd-backfill-space/6kdata 2020-09-12T14:29:52.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:52.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj518 td/osd-backfill-space/6kdata 2020-09-12T14:29:52.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:52.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj519 td/osd-backfill-space/6kdata 2020-09-12T14:29:52.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:52.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj520 td/osd-backfill-space/6kdata 2020-09-12T14:29:52.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:52.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj521 td/osd-backfill-space/6kdata 2020-09-12T14:29:52.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:52.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj522 td/osd-backfill-space/6kdata 2020-09-12T14:29:52.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:52.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj523 td/osd-backfill-space/6kdata 2020-09-12T14:29:52.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:52.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj524 td/osd-backfill-space/6kdata 2020-09-12T14:29:52.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:52.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj525 td/osd-backfill-space/6kdata 2020-09-12T14:29:52.997 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:52.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj526 td/osd-backfill-space/6kdata 2020-09-12T14:29:53.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:53.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj527 td/osd-backfill-space/6kdata 2020-09-12T14:29:53.240 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:53.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj528 td/osd-backfill-space/6kdata 2020-09-12T14:29:53.366 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:53.366 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj529 td/osd-backfill-space/6kdata 2020-09-12T14:29:53.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:53.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj530 td/osd-backfill-space/6kdata 2020-09-12T14:29:53.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:53.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj531 td/osd-backfill-space/6kdata 2020-09-12T14:29:53.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:53.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj532 td/osd-backfill-space/6kdata 2020-09-12T14:29:53.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:53.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj533 td/osd-backfill-space/6kdata 2020-09-12T14:29:54.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:54.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj534 td/osd-backfill-space/6kdata 2020-09-12T14:29:54.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:54.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj535 td/osd-backfill-space/6kdata 2020-09-12T14:29:54.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:54.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj536 td/osd-backfill-space/6kdata 2020-09-12T14:29:54.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:54.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj537 td/osd-backfill-space/6kdata 2020-09-12T14:29:54.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:54.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj538 td/osd-backfill-space/6kdata 2020-09-12T14:29:54.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:54.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj539 td/osd-backfill-space/6kdata 2020-09-12T14:29:54.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:54.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj540 td/osd-backfill-space/6kdata 2020-09-12T14:29:54.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:54.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj541 td/osd-backfill-space/6kdata 2020-09-12T14:29:55.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:55.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj542 td/osd-backfill-space/6kdata 2020-09-12T14:29:55.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:55.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj543 td/osd-backfill-space/6kdata 2020-09-12T14:29:55.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:55.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj544 td/osd-backfill-space/6kdata 2020-09-12T14:29:55.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:55.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj545 td/osd-backfill-space/6kdata 2020-09-12T14:29:55.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:55.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj546 td/osd-backfill-space/6kdata 2020-09-12T14:29:55.669 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:55.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj547 td/osd-backfill-space/6kdata 2020-09-12T14:29:55.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:55.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj548 td/osd-backfill-space/6kdata 2020-09-12T14:29:55.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:55.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj549 td/osd-backfill-space/6kdata 2020-09-12T14:29:56.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:56.315 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj550 td/osd-backfill-space/6kdata 2020-09-12T14:29:56.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:56.466 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj551 td/osd-backfill-space/6kdata 2020-09-12T14:29:56.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:56.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj552 td/osd-backfill-space/6kdata 2020-09-12T14:29:56.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:56.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj553 td/osd-backfill-space/6kdata 2020-09-12T14:29:56.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:56.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj554 td/osd-backfill-space/6kdata 2020-09-12T14:29:56.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:56.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj555 td/osd-backfill-space/6kdata 2020-09-12T14:29:57.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:57.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj556 td/osd-backfill-space/6kdata 2020-09-12T14:29:57.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:57.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj557 td/osd-backfill-space/6kdata 2020-09-12T14:29:57.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:57.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj558 td/osd-backfill-space/6kdata 2020-09-12T14:29:57.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:57.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj559 td/osd-backfill-space/6kdata 2020-09-12T14:29:57.653 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:57.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj560 td/osd-backfill-space/6kdata 2020-09-12T14:29:57.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:57.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj561 td/osd-backfill-space/6kdata 2020-09-12T14:29:57.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:57.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj562 td/osd-backfill-space/6kdata 2020-09-12T14:29:58.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:58.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj563 td/osd-backfill-space/6kdata 2020-09-12T14:29:58.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:58.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj564 td/osd-backfill-space/6kdata 2020-09-12T14:29:58.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:58.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj565 td/osd-backfill-space/6kdata 2020-09-12T14:29:58.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:58.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj566 td/osd-backfill-space/6kdata 2020-09-12T14:29:58.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:58.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj567 td/osd-backfill-space/6kdata 2020-09-12T14:29:58.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:58.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj568 td/osd-backfill-space/6kdata 2020-09-12T14:29:58.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:58.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj569 td/osd-backfill-space/6kdata 2020-09-12T14:29:58.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:58.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj570 td/osd-backfill-space/6kdata 2020-09-12T14:29:59.085 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:59.085 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj571 td/osd-backfill-space/6kdata 2020-09-12T14:29:59.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:59.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj572 td/osd-backfill-space/6kdata 2020-09-12T14:29:59.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:59.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj573 td/osd-backfill-space/6kdata 2020-09-12T14:29:59.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:59.454 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj574 td/osd-backfill-space/6kdata 2020-09-12T14:29:59.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:59.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj575 td/osd-backfill-space/6kdata 2020-09-12T14:29:59.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:59.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj576 td/osd-backfill-space/6kdata 2020-09-12T14:29:59.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:59.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj577 td/osd-backfill-space/6kdata 2020-09-12T14:29:59.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:29:59.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj578 td/osd-backfill-space/6kdata 2020-09-12T14:30:00.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:30:00.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj579 td/osd-backfill-space/6kdata 2020-09-12T14:30:00.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:30:00.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj580 td/osd-backfill-space/6kdata 2020-09-12T14:30:00.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:30:00.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj581 td/osd-backfill-space/6kdata 2020-09-12T14:30:00.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:30:00.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj582 td/osd-backfill-space/6kdata 2020-09-12T14:30:00.617 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:30:00.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj583 td/osd-backfill-space/6kdata 2020-09-12T14:30:00.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:30:00.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj584 td/osd-backfill-space/6kdata 2020-09-12T14:30:00.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:30:00.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj585 td/osd-backfill-space/6kdata 2020-09-12T14:30:01.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:30:01.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj586 td/osd-backfill-space/6kdata 2020-09-12T14:30:01.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:30:01.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj587 td/osd-backfill-space/6kdata 2020-09-12T14:30:01.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:30:01.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj588 td/osd-backfill-space/6kdata 2020-09-12T14:30:01.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:30:01.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj589 td/osd-backfill-space/6kdata 2020-09-12T14:30:01.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:30:01.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj590 td/osd-backfill-space/6kdata 2020-09-12T14:30:01.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:30:01.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj591 td/osd-backfill-space/6kdata 2020-09-12T14:30:01.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:30:01.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj592 td/osd-backfill-space/6kdata 2020-09-12T14:30:01.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:30:01.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj593 td/osd-backfill-space/6kdata 2020-09-12T14:30:02.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:30:02.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj594 td/osd-backfill-space/6kdata 2020-09-12T14:30:02.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:30:02.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj595 td/osd-backfill-space/6kdata 2020-09-12T14:30:02.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:30:02.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj596 td/osd-backfill-space/6kdata 2020-09-12T14:30:02.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:30:02.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj597 td/osd-backfill-space/6kdata 2020-09-12T14:30:02.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:30:02.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj598 td/osd-backfill-space/6kdata 2020-09-12T14:30:02.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:30:02.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj599 td/osd-backfill-space/6kdata 2020-09-12T14:30:02.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:598: TEST_backfill_grow: for i in $(seq $(expr $rmobjects + 1) $objects) 2020-09-12T14:30:02.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:600: TEST_backfill_grow: rados -p test put obj600 td/osd-backfill-space/6kdata 2020-09-12T14:30:02.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:603: TEST_backfill_grow: activate_osd td/osd-backfill-space 0 2020-09-12T14:30:02.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/osd-backfill-space 2020-09-12T14:30:02.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T14:30:02.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=0 2020-09-12T14:30:02.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T14:30:02.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/osd-backfill-space/0 2020-09-12T14:30:02.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:30:02.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:30:02.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:30:02.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:30:02.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/osd-backfill-space/0' 2020-09-12T14:30:02.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/0/journal' 2020-09-12T14:30:02.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T14:30:02.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T14:30:02.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:30:02.950 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T14:30:02.951 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:30:02.951 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:30:02.951 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:30:02.951 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:30:02.952 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:30:02.952 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:30:02.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:30:02.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:30:02.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:30:02.953 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:30:02.953 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:30:02.953 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:30:02.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:30:02.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T14:30:02.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T14:30:02.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/osd-backfill-space/0 2020-09-12T14:30:02.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.0 2020-09-12T14:30:02.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 0 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/0 --osd-journal=td/osd-backfill-space/0/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:30:02.956 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/osd-backfill-space/0/whoami 2020-09-12T14:30:02.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 0 = 0 ']' 2020-09-12T14:30:02.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T14:30:02.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T14:30:02.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T14:30:02.957 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.0 2020-09-12T14:30:02.975 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:30:02.974+0000 7fa5fd4ecec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:30:02.976 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:30:02.976+0000 7fa5fd4ecec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:30:02.978 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:30:02.977+0000 7fa5fd4ecec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:30:03.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: wait_for_osd up 0 2020-09-12T14:30:03.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:30:03.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T14:30:03.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:30:03.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:30:03.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:30:03.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:30:03.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:30:03.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:30:03.361 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:30:03.498 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:30:03.497+0000 7fa5fd4ecec0 -1 Falling back to public interface 2020-09-12T14:30:03.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:30:04.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:30:04.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:30:04.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:30:04.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:30:04.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:30:04.799 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:30:04.845 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:30:04.844+0000 7fa5fd4ecec0 -1 osd.0 24 log_to_monitors {default=true} 2020-09-12T14:30:05.241 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 28 up_thru 0 down_at 26 last_clean_interval [5,24) [v2:127.0.0.1:6802/173588,v1:127.0.0.1:6803/173588] [v2:127.0.0.1:6804/173588,v1:127.0.0.1:6805/173588] exists,up eaea4ad0-08c4-4433-ada9-0903fe013c91 2020-09-12T14:30:05.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:30:05.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:30:05.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:30:05.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:605: TEST_backfill_grow: ceph tell osd.1 debug kick_recovery_wq 0 2020-09-12T14:30:05.379 INFO:tasks.workunit.client.0.smithi027.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2020-09-12T14:30:05.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:607: TEST_backfill_grow: sleep 2 2020-09-12T14:30:07.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:609: TEST_backfill_grow: wait_for_clean 2020-09-12T14:30:07.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:30:07.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:30:07.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:30:07.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:30:07.390 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 90 .1 2020-09-12T14:30:07.391 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:30:07.391 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:30:07.391 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:30:07.391 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:30:07.391 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:30:07.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:30:07.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:30:07.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:30:07.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:30:07.497 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:30:07.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:30:07.909 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:30:07.909 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T14:30:07.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:30:07.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:30:07.910 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:30:08.062 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=120259084291 2020-09-12T14:30:08.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 120259084291 2020-09-12T14:30:08.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-120259084291' 2020-09-12T14:30:08.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:30:08.063 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:30:08.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949673001 2020-09-12T14:30:08.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949673001 2020-09-12T14:30:08.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-120259084291 1-42949673001' 2020-09-12T14:30:08.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:30:08.208 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:30:08.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509480 2020-09-12T14:30:08.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509480 2020-09-12T14:30:08.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-120259084291 1-42949673001 2-64424509480' 2020-09-12T14:30:08.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:30:08.351 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-120259084291 2020-09-12T14:30:08.358 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:30:08.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:30:08.358 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-120259084291 2020-09-12T14:30:08.358 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:30:08.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=120259084291 2020-09-12T14:30:08.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 120259084291' 2020-09-12T14:30:08.359 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:30:08.359 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 120259084291 2020-09-12T14:30:08.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 1 -lt 120259084291 2020-09-12T14:30:08.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:30:09.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:30:09.758 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:30:10.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 120259084291 -lt 120259084291 2020-09-12T14:30:10.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:30:10.182 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949673001 2020-09-12T14:30:10.182 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:30:10.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:30:10.183 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949673001 2020-09-12T14:30:10.183 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:30:10.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949673001 2020-09-12T14:30:10.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949673001' 2020-09-12T14:30:10.185 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949673001 2020-09-12T14:30:10.185 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:30:10.606 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949673001 -lt 42949673001 2020-09-12T14:30:10.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:30:10.607 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509480 2020-09-12T14:30:10.607 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:30:10.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:30:10.608 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509480 2020-09-12T14:30:10.608 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:30:10.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509480 2020-09-12T14:30:10.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509480' 2020-09-12T14:30:10.609 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509480 2020-09-12T14:30:10.610 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:30:11.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509480 -lt 64424509480 2020-09-12T14:30:11.027 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:30:11.027 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:11.027 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:11.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 1 == 0 2020-09-12T14:30:11.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:11.567 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:11.567 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:11.567 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:11.568 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:11.568 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:11.568 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:11.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:11.960 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:11.960 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:11.960 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:12.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:12.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' -1 2020-09-12T14:30:12.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1598: wait_for_clean: loop=0 2020-09-12T14:30:12.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1599: wait_for_clean: num_active_clean=0 2020-09-12T14:30:12.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:12.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:12.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:12.606 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:12.606 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:12.606 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:12.606 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:12.606 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:12.607 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:12.607 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:13.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:13.006 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:13.007 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:13.007 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:13.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:13.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:13.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:13.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:13.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:13.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:13.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:13.538 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:13.538 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:14.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=16912 2020-09-12T14:30:14.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 16912 '!=' null 2020-09-12T14:30:14.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:14.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:14.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:14.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:14.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:14.171 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:14.171 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:14.171 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:14.172 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:14.172 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:14.172 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:14.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:14.572 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:14.573 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:14.573 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:15.113 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:15.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:15.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:15.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:15.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:15.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:15.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:15.115 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:15.115 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:15.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=16912 2020-09-12T14:30:15.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 16912 '!=' null 2020-09-12T14:30:15.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:15.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:15.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:15.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:15.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:15.747 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:15.747 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:15.747 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:15.748 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:15.748 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:15.748 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:16.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:16.147 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:16.147 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:16.147 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:16.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:16.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:16.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:16.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:16.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:16.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:16.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:16.683 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:16.683 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:17.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=30749 2020-09-12T14:30:17.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 30749 '!=' null 2020-09-12T14:30:17.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:17.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:17.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:17.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:17.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:17.325 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:17.325 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:17.325 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:17.325 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:17.326 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:17.326 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:17.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:17.717 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:17.717 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:17.718 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:18.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:18.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:18.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:18.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:18.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:18.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:18.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:18.258 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:18.258 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:18.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=26359 2020-09-12T14:30:18.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 26359 '!=' null 2020-09-12T14:30:18.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:18.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:18.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:18.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:18.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:18.897 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:18.897 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:18.897 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:18.897 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:18.897 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:18.898 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:19.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:19.294 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:19.294 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:19.295 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:19.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:19.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:19.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:19.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:19.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:19.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:19.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:19.831 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:19.831 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:20.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=25087 2020-09-12T14:30:20.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 25087 '!=' null 2020-09-12T14:30:20.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:20.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:20.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:20.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:20.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:20.461 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:20.461 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:20.461 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:20.461 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:20.461 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:20.461 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:20.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:20.855 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:20.856 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:20.856 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:21.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:21.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:21.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:21.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:21.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:21.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:21.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:21.420 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:21.420 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:21.953 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=35327 2020-09-12T14:30:21.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 35327 '!=' null 2020-09-12T14:30:21.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:21.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:21.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:22.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:22.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:22.056 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:22.056 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:22.056 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:22.056 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:22.056 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:22.057 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:22.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:22.462 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:22.463 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:22.463 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:23.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:23.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:23.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:23.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:23.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:23.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:23.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:23.002 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:23.002 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:23.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=35327 2020-09-12T14:30:23.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 35327 '!=' null 2020-09-12T14:30:23.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:23.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:23.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:23.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:23.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:23.642 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:23.643 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:23.643 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:23.643 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:23.644 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:23.644 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:24.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:24.041 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:24.042 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:24.042 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:24.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:24.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:24.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:24.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:24.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:24.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:24.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:24.575 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:24.575 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:25.110 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=28159 2020-09-12T14:30:25.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 28159 '!=' null 2020-09-12T14:30:25.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:25.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:25.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:25.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:25.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:25.212 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:25.212 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:25.213 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:25.213 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:25.213 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:25.213 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:25.604 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:25.605 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:25.605 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:25.605 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:26.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:26.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:26.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:26.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:26.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:26.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:26.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:26.154 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:26.154 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:26.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=42494 2020-09-12T14:30:26.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 42494 '!=' null 2020-09-12T14:30:26.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:26.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:26.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:26.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:26.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:26.787 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:26.787 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:26.787 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:26.788 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:26.788 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:26.788 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:27.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:27.182 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:27.182 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:27.183 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:27.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:27.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:27.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:27.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:27.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:27.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:27.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:27.742 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:27.743 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:28.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=28671 2020-09-12T14:30:28.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 28671 '!=' null 2020-09-12T14:30:28.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:28.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:28.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:28.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:28.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:28.382 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:28.382 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:28.382 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:28.382 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:28.383 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:28.383 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:28.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:28.777 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:28.778 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:28.778 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:29.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:29.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:29.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:29.315 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:29.315 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:29.315 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:29.315 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:29.315 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:29.316 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:29.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=28671 2020-09-12T14:30:29.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 28671 '!=' null 2020-09-12T14:30:29.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:29.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:29.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:29.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:29.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:29.951 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:29.951 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:29.951 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:29.951 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:29.951 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:29.951 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:30.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:30.341 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:30.342 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:30.342 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:30.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:30.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:30.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:30.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:30.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:30.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:30.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:30.906 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:30.906 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:31.439 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=28671 2020-09-12T14:30:31.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 28671 '!=' null 2020-09-12T14:30:31.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:31.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:31.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:31.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:31.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:31.541 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:31.541 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:31.542 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:31.542 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:31.542 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:31.542 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:31.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:31.937 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:31.938 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:31.938 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:32.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:32.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:32.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:32.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:32.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:32.477 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:32.477 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:32.477 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:32.477 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:33.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=42494 2020-09-12T14:30:33.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 42494 '!=' null 2020-09-12T14:30:33.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:33.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:33.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:33.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:33.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:33.121 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:33.121 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:33.121 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:33.121 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:33.121 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:33.122 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:33.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:33.514 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:33.515 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:33.515 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:34.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:34.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:34.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:34.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:34.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:34.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:34.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:34.058 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:34.058 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:34.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=28159 2020-09-12T14:30:34.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 28159 '!=' null 2020-09-12T14:30:34.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:34.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:34.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:34.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:34.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:34.703 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:34.703 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:34.703 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:34.703 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:34.704 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:34.704 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:35.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:35.097 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:35.098 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:35.098 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:35.626 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:35.626 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:35.626 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:35.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:35.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:35.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:35.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:35.627 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:35.628 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:36.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=42495 2020-09-12T14:30:36.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 42495 '!=' null 2020-09-12T14:30:36.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:36.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:36.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:36.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:36.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:36.308 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:36.308 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:36.308 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:36.309 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:36.309 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:36.309 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:36.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:36.705 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:36.706 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:36.706 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:37.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:37.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:37.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:37.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:37.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:37.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:37.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:37.245 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:37.245 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:37.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=28159 2020-09-12T14:30:37.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 28159 '!=' null 2020-09-12T14:30:37.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:37.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:37.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:37.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:37.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:37.920 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:37.920 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:37.921 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:37.921 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:37.921 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:37.921 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:38.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:38.311 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:38.312 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:38.312 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:38.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:38.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:38.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:38.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:38.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:38.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:38.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:38.855 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:38.855 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:39.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=28159 2020-09-12T14:30:39.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 28159 '!=' null 2020-09-12T14:30:39.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:39.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:39.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:39.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:39.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:39.490 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:39.490 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:39.490 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:39.490 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:39.490 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:39.490 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:39.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:39.884 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:39.884 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:39.884 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:40.434 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:40.434 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:40.434 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:40.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:40.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:40.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:40.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:40.435 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:40.435 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:40.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=28159 2020-09-12T14:30:40.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 28159 '!=' null 2020-09-12T14:30:40.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:40.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:40.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:41.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:41.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:41.072 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:41.072 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:41.072 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:41.072 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:41.072 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:41.073 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:41.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:41.471 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:41.472 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:41.472 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:42.014 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:42.014 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:42.014 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:42.014 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:42.014 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:42.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:42.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:42.015 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:42.015 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:42.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=42495 2020-09-12T14:30:42.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 42495 '!=' null 2020-09-12T14:30:42.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:42.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:42.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:42.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:42.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:42.655 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:42.655 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:42.655 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:42.655 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:42.655 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:42.656 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:43.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:43.049 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:43.049 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:43.049 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:43.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:43.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:43.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:43.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:43.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:43.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:43.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:43.582 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:43.582 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:44.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=28671 2020-09-12T14:30:44.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 28671 '!=' null 2020-09-12T14:30:44.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:44.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:44.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:44.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:44.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:44.216 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:44.216 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:44.217 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:44.217 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:44.217 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:44.217 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:44.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:44.611 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:44.612 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:44.612 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:45.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:45.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:45.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:45.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:45.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:45.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:45.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:45.150 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:45.150 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:45.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=28671 2020-09-12T14:30:45.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 28671 '!=' null 2020-09-12T14:30:45.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:45.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:45.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:45.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:45.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:45.807 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:45.807 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:45.807 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:45.807 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:45.808 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:45.808 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:46.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:46.204 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:46.204 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:46.205 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:46.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:46.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:46.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:46.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:46.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:46.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:46.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:46.741 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:46.741 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:47.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=44031 2020-09-12T14:30:47.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 44031 '!=' null 2020-09-12T14:30:47.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:47.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:47.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:47.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:47.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:47.373 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:47.373 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:47.374 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:47.374 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:47.374 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:47.374 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:47.767 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:47.768 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:47.768 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:47.768 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:48.298 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:48.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:48.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:48.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:48.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:48.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:48.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:48.300 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:48.300 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:48.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=29695 2020-09-12T14:30:48.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 29695 '!=' null 2020-09-12T14:30:48.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:48.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:48.841 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:48.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:48.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:48.942 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:48.942 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:48.942 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:48.942 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:48.942 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:48.942 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:49.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:49.333 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:49.333 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:49.333 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:49.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:49.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:49.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:49.864 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:49.864 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:49.864 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:49.864 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:49.864 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:49.864 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:50.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=29695 2020-09-12T14:30:50.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 29695 '!=' null 2020-09-12T14:30:50.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:50.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:50.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:50.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:50.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:50.512 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:50.512 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:50.512 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:50.512 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:50.512 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:50.513 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:50.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:50.913 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:50.913 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:50.913 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:51.447 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:51.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:51.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:51.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:51.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:51.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:51.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:51.450 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:51.450 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:51.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=44032 2020-09-12T14:30:51.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 44032 '!=' null 2020-09-12T14:30:51.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:51.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:51.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:52.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:52.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:52.088 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:52.088 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:52.088 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:52.088 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:52.089 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:52.089 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:52.481 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:52.481 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:52.481 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:52.482 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:53.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:53.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:53.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:53.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:53.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:53.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:53.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:53.019 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:53.020 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:53.558 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=44032 2020-09-12T14:30:53.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 44032 '!=' null 2020-09-12T14:30:53.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:53.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:53.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:53.659 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:53.659 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:53.660 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:53.660 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:53.660 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:53.661 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:53.661 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:53.661 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:54.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:54.065 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:54.066 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:54.066 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:54.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:54.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:54.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:54.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:54.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:54.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:54.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:54.602 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:54.603 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:55.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=29695 2020-09-12T14:30:55.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 29695 '!=' null 2020-09-12T14:30:55.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:55.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:55.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:55.236 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:55.236 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:55.237 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:55.237 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:55.237 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:55.237 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:55.237 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:55.238 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:55.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:55.633 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:55.633 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:55.633 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:56.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:56.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:56.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:56.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:56.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:56.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:56.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:56.205 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:56.205 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:56.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=45055 2020-09-12T14:30:56.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 45055 '!=' null 2020-09-12T14:30:56.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:56.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:56.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:56.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:56.848 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:56.848 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:56.848 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:56.848 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:56.848 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:56.848 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:56.849 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:57.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:57.245 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:57.245 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:57.246 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:57.782 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:57.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:57.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:57.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:57.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:57.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:57.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:57.784 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:57.784 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:58.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=29695 2020-09-12T14:30:58.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 29695 '!=' null 2020-09-12T14:30:58.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:58.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:58.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:58.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:58.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:58.414 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:58.414 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:58.415 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:58.415 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:58.415 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:58.415 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:30:58.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:30:58.810 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:30:58.810 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:30:58.810 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:30:59.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:30:59.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:30:59.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:30:59.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:30:59.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:30:59.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:30:59.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:30:59.341 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:30:59.341 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:30:59.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=29695 2020-09-12T14:30:59.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 29695 '!=' null 2020-09-12T14:30:59.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:30:59.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:30:59.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:30:59.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:30:59.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:30:59.976 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:30:59.976 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:30:59.976 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:30:59.976 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:30:59.977 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:30:59.977 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:31:00.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:31:00.372 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:31:00.373 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:31:00.373 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:31:00.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:31:00.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:31:00.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:31:00.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:31:00.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:31:00.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:31:00.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:31:00.925 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:31:00.925 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:31:01.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=29695 2020-09-12T14:31:01.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 29695 '!=' null 2020-09-12T14:31:01.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:31:01.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:31:01.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:31:01.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:31:01.557 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:31:01.558 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:31:01.558 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:31:01.558 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:31:01.558 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:31:01.558 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:31:01.558 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:31:01.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:31:01.972 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:31:01.973 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:31:01.973 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:31:02.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:31:02.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:31:02.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:31:02.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:31:02.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:31:02.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:31:02.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:31:02.505 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:31:02.505 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:31:03.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=43519 2020-09-12T14:31:03.035 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 43519 '!=' null 2020-09-12T14:31:03.035 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:31:03.035 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:31:03.035 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:31:03.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:31:03.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:31:03.136 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:31:03.136 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:31:03.136 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:31:03.136 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:31:03.137 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:31:03.137 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:31:03.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:31:03.528 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:31:03.528 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:31:03.528 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:31:04.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:31:04.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:31:04.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:31:04.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:31:04.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:31:04.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:31:04.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:31:04.070 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:31:04.070 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:31:04.597 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=29183 2020-09-12T14:31:04.597 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 29183 '!=' null 2020-09-12T14:31:04.597 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:31:04.598 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:31:04.598 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:31:04.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:31:04.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:31:04.699 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:31:04.699 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:31:04.699 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:31:04.699 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:31:04.699 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:31:04.699 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:31:05.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:31:05.105 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:31:05.106 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:31:05.106 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:31:05.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:31:05.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:31:05.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:31:05.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:31:05.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:31:05.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:31:05.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:31:05.642 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:31:05.642 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:31:06.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=44544 2020-09-12T14:31:06.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 44544 '!=' null 2020-09-12T14:31:06.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:31:06.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:31:06.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:31:06.316 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:31:06.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:31:06.317 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:31:06.317 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:31:06.317 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:31:06.317 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:31:06.317 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:31:06.318 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:31:06.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:31:06.710 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:31:06.711 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:31:06.711 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:31:07.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:31:07.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:31:07.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:31:07.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:31:07.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:31:07.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:31:07.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:31:07.239 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:31:07.239 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:31:07.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=44544 2020-09-12T14:31:07.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 44544 '!=' null 2020-09-12T14:31:07.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:31:07.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:31:07.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:31:07.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:31:07.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:31:07.882 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:31:07.882 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:31:07.883 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:31:07.883 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:31:07.883 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:31:07.883 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:31:08.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:31:08.279 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:31:08.279 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:31:08.280 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:31:08.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:31:08.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:31:08.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:31:08.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:31:08.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:31:08.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:31:08.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:31:08.813 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:31:08.813 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:31:09.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=29183 2020-09-12T14:31:09.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 29183 '!=' null 2020-09-12T14:31:09.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:31:09.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:31:09.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:31:09.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:31:09.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:31:09.450 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:31:09.450 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:31:09.450 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:31:09.450 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:31:09.450 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:31:09.450 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:31:09.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:31:09.841 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:31:09.841 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:31:09.841 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:31:10.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:31:10.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:31:10.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:31:10.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:31:10.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:31:10.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:31:10.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:31:10.392 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:31:10.392 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:31:10.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=29183 2020-09-12T14:31:10.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 29183 '!=' null 2020-09-12T14:31:10.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:31:10.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:31:10.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:31:11.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:31:11.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:31:11.025 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:31:11.025 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:31:11.025 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:31:11.025 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:31:11.025 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:31:11.026 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:31:11.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:31:11.427 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:31:11.428 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:31:11.428 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:31:11.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:31:11.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:31:11.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:31:11.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:31:11.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:31:11.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:31:11.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:31:11.969 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:31:11.969 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:31:12.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=42495 2020-09-12T14:31:12.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 42495 '!=' null 2020-09-12T14:31:12.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:31:12.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:31:12.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:31:12.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:31:12.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:31:12.596 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:31:12.596 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:31:12.596 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:31:12.596 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:31:12.597 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:31:12.597 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:31:12.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:31:12.989 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:31:12.990 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:31:12.990 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:31:13.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:31:13.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:31:13.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:31:13.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:31:13.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:31:13.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:31:13.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:31:13.528 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:31:13.528 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:31:14.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=28671 2020-09-12T14:31:14.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 28671 '!=' null 2020-09-12T14:31:14.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:31:14.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:31:14.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:31:14.165 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:31:14.165 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:31:14.166 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:31:14.166 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:31:14.166 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:31:14.166 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:31:14.166 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:31:14.167 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:31:14.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:31:14.557 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:31:14.557 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:31:14.557 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:31:15.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:31:15.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:31:15.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:31:15.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:31:15.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:31:15.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:31:15.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:31:15.102 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:31:15.103 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:31:15.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=28671 2020-09-12T14:31:15.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 28671 '!=' null 2020-09-12T14:31:15.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:31:15.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:31:15.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:31:15.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:31:15.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:31:15.737 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:31:15.737 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:31:15.737 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:31:15.738 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:31:15.738 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:31:15.738 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:31:16.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:31:16.156 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:31:16.157 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:31:16.157 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:31:16.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:31:16.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:31:16.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:31:16.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:31:16.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:31:16.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:31:16.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:31:16.692 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:31:16.692 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:31:17.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=44543 2020-09-12T14:31:17.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 44543 '!=' null 2020-09-12T14:31:17.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:31:17.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:31:17.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:31:17.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:31:17.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:31:17.330 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:31:17.330 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:31:17.331 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:31:17.331 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:31:17.331 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:31:17.331 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:31:17.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:31:17.734 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:31:17.734 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:31:17.734 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:31:18.264 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:31:18.264 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:31:18.264 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:31:18.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:31:18.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:31:18.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:31:18.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:31:18.265 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:31:18.265 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:31:18.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=29182 2020-09-12T14:31:18.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 29182 '!=' null 2020-09-12T14:31:18.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:31:18.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:31:18.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:31:18.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:31:18.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:31:18.907 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:31:18.907 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:31:18.907 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:31:18.908 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:31:18.908 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:31:18.908 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:31:19.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:31:19.302 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:31:19.302 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:31:19.302 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:31:19.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:31:19.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:31:19.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:31:19.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:31:19.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:31:19.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:31:19.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:31:19.835 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:31:19.836 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:31:20.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=29183 2020-09-12T14:31:20.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 29183 '!=' null 2020-09-12T14:31:20.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:31:20.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:31:20.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:31:20.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:31:20.468 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:31:20.468 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:31:20.469 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:31:20.469 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:31:20.469 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:31:20.469 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:31:20.469 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:31:20.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:31:20.861 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:31:20.862 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:31:20.862 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:31:21.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:31:21.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:31:21.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:31:21.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:31:21.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:31:21.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:31:21.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:31:21.451 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:31:21.451 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:31:21.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=43007 2020-09-12T14:31:21.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 43007 '!=' null 2020-09-12T14:31:21.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:31:21.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:31:21.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:31:22.086 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:31:22.086 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:31:22.087 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:31:22.087 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:31:22.087 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:31:22.087 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:31:22.087 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:31:22.087 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:31:22.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:31:22.486 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:31:22.487 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:31:22.487 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:31:23.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:31:23.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:31:23.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:31:23.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:31:23.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:31:23.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:31:23.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:31:23.029 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:31:23.030 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:31:23.558 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=43007 2020-09-12T14:31:24.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 43007 '!=' null 2020-09-12T14:31:24.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:31:24.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:31:24.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:31:24.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:31:24.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:31:24.056 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:31:24.057 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:31:24.057 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:31:24.057 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:31:24.057 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:31:24.057 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:31:24.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:31:24.057 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:31:24.058 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:31:24.058 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:31:24.590 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:31:24.590 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:31:24.590 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:31:24.590 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:31:24.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:31:24.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:31:24.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:31:24.591 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:31:24.591 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:31:25.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=29694 2020-09-12T14:31:25.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 29694 '!=' null 2020-09-12T14:31:25.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:31:25.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:31:25.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:31:25.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:31:25.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:31:25.239 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:31:25.239 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:31:25.240 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:31:25.240 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:31:25.240 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:31:25.240 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:31:25.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:31:25.638 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:31:25.638 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:31:25.638 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:31:26.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:31:26.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:31:26.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:31:26.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:31:26.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:31:26.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:31:26.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:31:26.175 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:31:26.175 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:31:26.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=44542 2020-09-12T14:31:26.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 44542 '!=' null 2020-09-12T14:31:26.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:31:26.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:31:26.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:31:26.859 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:31:26.859 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:31:26.860 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:31:26.860 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:31:26.860 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:31:26.861 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:31:26.861 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:31:26.861 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:31:27.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:31:27.252 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:31:27.253 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:31:27.253 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:31:27.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:31:27.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:31:27.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:31:27.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:31:27.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:31:27.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:31:27.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:31:27.791 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:31:27.791 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:31:28.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=43005 2020-09-12T14:31:28.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 43005 '!=' null 2020-09-12T14:31:28.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:31:28.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:31:28.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:31:28.433 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:31:28.433 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:31:28.434 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:31:28.434 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:31:28.434 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:31:28.435 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:31:28.435 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:31:28.435 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:31:28.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:31:28.825 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:31:28.826 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:31:28.826 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:31:29.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:31:29.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:31:29.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:31:29.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:31:29.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:31:29.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:31:29.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:31:29.359 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:31:29.360 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:31:29.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=22270 2020-09-12T14:31:29.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 22270 '!=' null 2020-09-12T14:31:29.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:31:29.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:31:29.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:31:30.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:31:30.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:31:30.020 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:31:30.020 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:31:30.020 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:31:30.020 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:31:30.020 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:31:30.021 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:31:30.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=0 2020-09-12T14:31:30.415 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:31:30.416 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:31:30.416 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:31:30.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 0 = 1 2020-09-12T14:31:30.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1597: wait_for_clean: test 0 '!=' 0 2020-09-12T14:31:30.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1600: wait_for_clean: get_is_making_recovery_progress 2020-09-12T14:31:30.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: get_is_making_recovery_progress: local recovery_progress 2020-09-12T14:31:30.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1273: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2020-09-12T14:31:30.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2020-09-12T14:31:30.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1275: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2020-09-12T14:31:30.950 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: ceph --format json status 2020-09-12T14:31:30.950 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2020-09-12T14:31:31.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: get_is_making_recovery_progress: local progress=22270 2020-09-12T14:31:31.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1278: get_is_making_recovery_progress: test 22270 '!=' null 2020-09-12T14:31:31.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1601: wait_for_clean: loop=0 2020-09-12T14:31:31.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1607: wait_for_clean: eval 2020-09-12T14:31:31.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1608: wait_for_clean: sleep 0.1 2020-09-12T14:31:31.584 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1609: wait_for_clean: loop+=1 2020-09-12T14:31:31.584 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:31:31.585 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:31:31.585 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:31:31.585 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:31:31.585 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:31:31.585 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:31:31.586 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:31:32.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=1 2020-09-12T14:31:32.011 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:31:32.012 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:31:32.012 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:31:32.547 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 1 = 1 2020-09-12T14:31:32.547 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T14:31:32.547 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T14:31:32.547 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:611: TEST_backfill_grow: delete_pool test 2020-09-12T14:31:32.548 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: delete_pool: local poolname=test 2020-09-12T14:31:32.548 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2020-09-12T14:31:33.094 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test' does not exist 2020-09-12T14:31:33.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:612: TEST_backfill_grow: kill_daemons td/osd-backfill-space 2020-09-12T14:31:33.104 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:31:33.104 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:31:33.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:31:33.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:31:33.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:31:33.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:31:33.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:613: TEST_backfill_grow: grep -q 'num_bytes mismatch' td/osd-backfill-space/osd.0.log td/osd-backfill-space/osd.1.log td/osd-backfill-space/osd.2.log 2020-09-12T14:31:33.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:38: run: teardown td/osd-backfill-space 2020-09-12T14:31:33.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-backfill-space 2020-09-12T14:31:33.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:31:33.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-backfill-space KILL 2020-09-12T14:31:33.243 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:31:33.243 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:31:33.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:31:33.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:31:33.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:31:33.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:31:33.248 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:31:33.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:31:33.249 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:31:33.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:31:33.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:31:33.250 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:31:33.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:31:33.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:31:33.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:31:33.252 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:31:33.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:31:33.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:31:33.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-backfill-space 2020-09-12T14:31:33.290 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:31:33.291 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:31:33.291 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:31:33.291 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.145692 2020-09-12T14:31:33.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:31:33.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:31:33.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:35: run: for func in $funcs 2020-09-12T14:31:33.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:36: run: setup td/osd-backfill-space 2020-09-12T14:31:33.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-backfill-space 2020-09-12T14:31:33.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-backfill-space 2020-09-12T14:31:33.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-backfill-space 2020-09-12T14:31:33.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:31:33.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-backfill-space KILL 2020-09-12T14:31:33.294 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:31:33.294 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:31:33.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:31:33.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:31:33.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:31:33.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:31:33.295 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:31:33.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:31:33.295 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:31:33.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:31:33.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:31:33.296 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:31:33.298 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:31:33.298 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:31:33.298 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:31:33.299 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:31:33.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:31:33.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:31:33.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-backfill-space 2020-09-12T14:31:33.301 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:31:33.301 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:31:33.301 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:31:33.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.145692 2020-09-12T14:31:33.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:31:33.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:31:33.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-backfill-space 2020-09-12T14:31:33.303 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2020-09-12T14:31:33.303 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:31:33.303 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:31:33.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.145692 2020-09-12T14:31:33.305 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2020-09-12T14:31:33.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2020-09-12T14:31:33.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2020-09-12T14:31:33.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-backfill-space 1' TERM HUP INT 2020-09-12T14:31:33.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:37: run: TEST_backfill_multi_partial td/osd-backfill-space 2020-09-12T14:31:33.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:429: TEST_backfill_multi_partial: local dir=td/osd-backfill-space 2020-09-12T14:31:33.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:430: TEST_backfill_multi_partial: local EC= 2020-09-12T14:31:33.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:431: TEST_backfill_multi_partial: local pools=2 2020-09-12T14:31:33.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:432: TEST_backfill_multi_partial: local OSDS=3 2020-09-12T14:31:33.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:434: TEST_backfill_multi_partial: run_mon td/osd-backfill-space a 2020-09-12T14:31:33.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-backfill-space 2020-09-12T14:31:33.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2020-09-12T14:31:33.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2020-09-12T14:31:33.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2020-09-12T14:31:33.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-backfill-space/a 2020-09-12T14:31:33.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-backfill-space/a --run-dir=td/osd-backfill-space 2020-09-12T14:31:33.604 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2020-09-12T14:31:33.605 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:31:33.605 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:31:33.605 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:31:33.605 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:31:33.605 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:31:33.605 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:31:33.606 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-backfill-space/a '--log-file=td/osd-backfill-space/$name.log' '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --mon-cluster-log-file=td/osd-backfill-space/log --run-dir=td/osd-backfill-space '--pid-file=td/osd-backfill-space/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 2020-09-12T14:31:33.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: cat 2020-09-12T14:31:33.765 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a fsid 2020-09-12T14:31:33.765 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:31:33.765 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:31:33.765 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=fsid 2020-09-12T14:31:33.765 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .fsid 2020-09-12T14:31:33.766 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:31:33.766 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:31:33.767 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:31:33.767 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:31:33.767 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:31:33.767 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:31:33.767 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.145692/ceph-mon.a.asok 2020-09-12T14:31:33.767 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:31:33.767 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.145692/ceph-mon.a.asok config get fsid 2020-09-12T14:31:33.874 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a mon_host 2020-09-12T14:31:33.874 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:31:33.874 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:31:33.874 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=mon_host 2020-09-12T14:31:33.874 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .mon_host 2020-09-12T14:31:33.875 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:31:33.875 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:31:33.875 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:31:33.876 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:31:33.876 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:31:33.876 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:31:33.876 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.145692/ceph-mon.a.asok 2020-09-12T14:31:33.876 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:31:33.876 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.145692/ceph-mon.a.asok config get mon_host 2020-09-12T14:31:33.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:435: TEST_backfill_multi_partial: run_mgr td/osd-backfill-space x 2020-09-12T14:31:33.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: local dir=td/osd-backfill-space 2020-09-12T14:31:33.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: shift 2020-09-12T14:31:33.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: local id=x 2020-09-12T14:31:33.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: shift 2020-09-12T14:31:33.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: local data=td/osd-backfill-space/x 2020-09-12T14:31:33.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:558: run_mgr: ceph config set mgr mgr/devicehealth/enable_monitoring off --force 2020-09-12T14:31:34.482 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: get_asok_path 2020-09-12T14:31:34.482 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:31:34.482 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:31:34.483 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:31:34.483 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:31:34.483 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:31:34.483 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:31:34.484 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:31:34.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-backfill-space/x '--log-file=td/osd-backfill-space/$name.log' '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --run-dir=td/osd-backfill-space '--pid-file=td/osd-backfill-space/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:31:34.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:436: TEST_backfill_multi_partial: export CEPH_ARGS 2020-09-12T14:31:34.507 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:438: TEST_backfill_multi_partial: expr 3 - 1 2020-09-12T14:31:34.509 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:438: TEST_backfill_multi_partial: seq 0 2 2020-09-12T14:31:34.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:438: TEST_backfill_multi_partial: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:31:34.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:440: TEST_backfill_multi_partial: run_osd td/osd-backfill-space 0 2020-09-12T14:31:34.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:31:34.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:31:34.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=0 2020-09-12T14:31:34.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:31:34.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/0 2020-09-12T14:31:34.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:31:34.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:31:34.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:31:34.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:31:34.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/0' 2020-09-12T14:31:34.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/0/journal' 2020-09-12T14:31:34.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:31:34.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:31:34.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:31:34.513 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:31:34.514 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:31:34.514 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:31:34.514 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:31:34.514 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:31:34.514 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:31:34.514 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:31:34.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:31:34.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:31:34.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:31:34.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:31:34.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:31:34.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:31:34.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:31:34.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:31:34.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:31:34.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:31:34.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:31:34.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/0 2020-09-12T14:31:34.516 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:31:34.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=7e9b1de6-c11a-430c-832e-977da2087fcb 2020-09-12T14:31:34.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd0 7e9b1de6-c11a-430c-832e-977da2087fcb' 2020-09-12T14:31:34.517 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:31:34.517 INFO:tasks.workunit.client.0.smithi027.stdout:add osd0 7e9b1de6-c11a-430c-832e-977da2087fcb 2020-09-12T14:31:34.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDG21xfVp1ZIBAAV2z2m3Lqek/oQN5XxoKvNQ== 2020-09-12T14:31:34.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDG21xfVp1ZIBAAV2z2m3Lqek/oQN5XxoKvNQ=="}' 2020-09-12T14:31:34.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 7e9b1de6-c11a-430c-832e-977da2087fcb -i td/osd-backfill-space/0/new.json 2020-09-12T14:31:34.892 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:31:34.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/0/new.json 2020-09-12T14:31:34.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 0 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/0 --osd-journal=td/osd-backfill-space/0/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDG21xfVp1ZIBAAV2z2m3Lqek/oQN5XxoKvNQ== --osd-uuid 7e9b1de6-c11a-430c-832e-977da2087fcb 2020-09-12T14:31:34.932 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:34.932+0000 7f1267311ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:31:34.934 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:34.934+0000 7f1267311ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:31:34.935 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:34.935+0000 7f1267311ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:31:34.935 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:34.935+0000 7f1267311ec0 -1 bluestore(td/osd-backfill-space/0/block) _read_bdev_label failed to open td/osd-backfill-space/0/block: (2) No such file or directory 2020-09-12T14:31:34.936 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:34.935+0000 7f1267311ec0 -1 bluestore(td/osd-backfill-space/0/block) _read_bdev_label failed to open td/osd-backfill-space/0/block: (2) No such file or directory 2020-09-12T14:31:34.936 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:34.935+0000 7f1267311ec0 -1 bluestore(td/osd-backfill-space/0/block) _read_bdev_label failed to open td/osd-backfill-space/0/block: (2) No such file or directory 2020-09-12T14:31:34.991 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:34.991+0000 7f1267311ec0 -1 bluestore(td/osd-backfill-space/0) _read_fsid unparsable uuid 2020-09-12T14:31:39.119 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/0/keyring 2020-09-12T14:31:39.119 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:31:39.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd0 key to auth repository 2020-09-12T14:31:39.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:31:39.120 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd0 key to auth repository 2020-09-12T14:31:39.669 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.0 2020-09-12T14:31:39.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:31:39.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:31:39.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:31:39.670 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.0 2020-09-12T14:31:39.671 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 0 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/0 --osd-journal=td/osd-backfill-space/0/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:31:39.695 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:39.694+0000 7f55dfb8bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:31:39.712 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:39.710+0000 7f55dfb8bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:31:39.714 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:39.713+0000 7f55dfb8bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:31:40.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 0 2020-09-12T14:31:40.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:31:40.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T14:31:40.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:31:40.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:31:40.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:31:40.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:31:40.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:31:40.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:31:40.077 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:31:40.242 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:40.241+0000 7f55dfb8bec0 -1 Falling back to public interface 2020-09-12T14:31:40.496 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:31:41.030 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:41.028+0000 7f55dfb8bec0 -1 osd.0 0 log_to_monitors {default=true} 2020-09-12T14:31:41.498 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:31:41.498 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:31:41.498 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:31:41.498 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:31:41.498 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:31:41.499 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:31:41.944 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/179338,v1:127.0.0.1:6803/179338] [v2:127.0.0.1:6804/179338,v1:127.0.0.1:6805/179338] exists,up 7e9b1de6-c11a-430c-832e-977da2087fcb 2020-09-12T14:31:41.944 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:31:41.944 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:31:41.944 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:31:41.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:438: TEST_backfill_multi_partial: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:31:41.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:440: TEST_backfill_multi_partial: run_osd td/osd-backfill-space 1 2020-09-12T14:31:41.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:31:41.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:31:41.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=1 2020-09-12T14:31:41.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:31:41.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/1 2020-09-12T14:31:41.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:31:41.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:31:41.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:31:41.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:31:41.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/1' 2020-09-12T14:31:41.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/1/journal' 2020-09-12T14:31:41.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:31:41.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:31:41.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:31:41.947 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:31:41.947 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:31:41.947 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:31:41.947 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:31:41.947 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:31:41.948 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:31:41.948 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:31:41.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:31:41.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:31:41.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:31:41.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:31:41.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:31:41.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:31:41.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:31:41.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:31:41.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:31:41.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:31:41.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:31:41.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/1 2020-09-12T14:31:41.951 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:31:41.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=ce656e6a-6430-4cc2-bd95-29130ea1369f 2020-09-12T14:31:41.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd1 ce656e6a-6430-4cc2-bd95-29130ea1369f' 2020-09-12T14:31:41.951 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:31:41.951 INFO:tasks.workunit.client.0.smithi027.stdout:add osd1 ce656e6a-6430-4cc2-bd95-29130ea1369f 2020-09-12T14:31:41.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDN21xfXSpWORAAux81DvZYcS+VzqVoysHMrQ== 2020-09-12T14:31:41.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDN21xfXSpWORAAux81DvZYcS+VzqVoysHMrQ=="}' 2020-09-12T14:31:41.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new ce656e6a-6430-4cc2-bd95-29130ea1369f -i td/osd-backfill-space/1/new.json 2020-09-12T14:31:42.486 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:31:42.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/1/new.json 2020-09-12T14:31:42.498 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 1 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/1 --osd-journal=td/osd-backfill-space/1/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDN21xfXSpWORAAux81DvZYcS+VzqVoysHMrQ== --osd-uuid ce656e6a-6430-4cc2-bd95-29130ea1369f 2020-09-12T14:31:42.526 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:42.524+0000 7f163773fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:31:42.527 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:42.526+0000 7f163773fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:31:42.528 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:42.527+0000 7f163773fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:31:42.528 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:42.527+0000 7f163773fec0 -1 bluestore(td/osd-backfill-space/1/block) _read_bdev_label failed to open td/osd-backfill-space/1/block: (2) No such file or directory 2020-09-12T14:31:42.529 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:42.527+0000 7f163773fec0 -1 bluestore(td/osd-backfill-space/1/block) _read_bdev_label failed to open td/osd-backfill-space/1/block: (2) No such file or directory 2020-09-12T14:31:42.529 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:42.527+0000 7f163773fec0 -1 bluestore(td/osd-backfill-space/1/block) _read_bdev_label failed to open td/osd-backfill-space/1/block: (2) No such file or directory 2020-09-12T14:31:42.561 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:42.560+0000 7f163773fec0 -1 bluestore(td/osd-backfill-space/1) _read_fsid unparsable uuid 2020-09-12T14:31:46.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/1/keyring 2020-09-12T14:31:46.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:31:46.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd1 key to auth repository 2020-09-12T14:31:46.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:31:46.519 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd1 key to auth repository 2020-09-12T14:31:47.062 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T14:31:47.062 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.1 2020-09-12T14:31:47.062 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 1 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/1 --osd-journal=td/osd-backfill-space/1/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:31:47.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:31:47.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:31:47.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:31:47.088 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:47.086+0000 7ff400fabec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:31:47.125 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:47.124+0000 7ff400fabec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:31:47.127 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:47.125+0000 7ff400fabec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:31:47.482 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 1 2020-09-12T14:31:47.482 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:31:47.482 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:31:47.482 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:31:47.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:31:47.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:31:47.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:31:47.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:31:47.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:31:47.484 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:31:47.647 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:47.645+0000 7ff400fabec0 -1 Falling back to public interface 2020-09-12T14:31:47.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:31:48.501 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:48.500+0000 7ff400fabec0 -1 osd.1 0 log_to_monitors {default=true} 2020-09-12T14:31:48.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:31:48.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:31:48.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:31:48.892 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:31:48.892 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:31:48.892 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:31:49.317 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/180332,v1:127.0.0.1:6811/180332] [v2:127.0.0.1:6812/180332,v1:127.0.0.1:6813/180332] exists,up ce656e6a-6430-4cc2-bd95-29130ea1369f 2020-09-12T14:31:49.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:31:49.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:31:49.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:31:49.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:438: TEST_backfill_multi_partial: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:31:49.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:440: TEST_backfill_multi_partial: run_osd td/osd-backfill-space 2 2020-09-12T14:31:49.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:31:49.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:31:49.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=2 2020-09-12T14:31:49.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:31:49.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/2 2020-09-12T14:31:49.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:31:49.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:31:49.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:31:49.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:31:49.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/2' 2020-09-12T14:31:49.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/2/journal' 2020-09-12T14:31:49.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:31:49.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:31:49.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:31:49.320 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:31:49.320 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:31:49.320 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:31:49.320 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:31:49.320 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:31:49.321 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:31:49.321 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:31:49.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:31:49.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:31:49.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:31:49.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:31:49.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:31:49.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:31:49.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:31:49.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:31:49.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:31:49.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:31:49.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:31:49.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/2 2020-09-12T14:31:49.324 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:31:49.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=bddb3624-216d-4188-90f9-94cad5cb9bd8 2020-09-12T14:31:49.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd2 bddb3624-216d-4188-90f9-94cad5cb9bd8' 2020-09-12T14:31:49.324 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:31:49.324 INFO:tasks.workunit.client.0.smithi027.stdout:add osd2 bddb3624-216d-4188-90f9-94cad5cb9bd8 2020-09-12T14:31:49.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDV21xfHn8LFBAAhIt5atCi7UwR94kN1Qw/pA== 2020-09-12T14:31:49.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDV21xfHn8LFBAAhIt5atCi7UwR94kN1Qw/pA=="}' 2020-09-12T14:31:49.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new bddb3624-216d-4188-90f9-94cad5cb9bd8 -i td/osd-backfill-space/2/new.json 2020-09-12T14:31:49.897 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:31:49.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/2/new.json 2020-09-12T14:31:49.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 2 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/2 --osd-journal=td/osd-backfill-space/2/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDV21xfHn8LFBAAhIt5atCi7UwR94kN1Qw/pA== --osd-uuid bddb3624-216d-4188-90f9-94cad5cb9bd8 2020-09-12T14:31:49.937 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:49.936+0000 7fc0fc64fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:31:49.939 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:49.937+0000 7fc0fc64fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:31:49.939 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:49.938+0000 7fc0fc64fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:31:49.940 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:49.939+0000 7fc0fc64fec0 -1 bluestore(td/osd-backfill-space/2/block) _read_bdev_label failed to open td/osd-backfill-space/2/block: (2) No such file or directory 2020-09-12T14:31:49.940 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:49.939+0000 7fc0fc64fec0 -1 bluestore(td/osd-backfill-space/2/block) _read_bdev_label failed to open td/osd-backfill-space/2/block: (2) No such file or directory 2020-09-12T14:31:49.940 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:49.939+0000 7fc0fc64fec0 -1 bluestore(td/osd-backfill-space/2/block) _read_bdev_label failed to open td/osd-backfill-space/2/block: (2) No such file or directory 2020-09-12T14:31:49.983 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:49.983+0000 7fc0fc64fec0 -1 bluestore(td/osd-backfill-space/2) _read_fsid unparsable uuid 2020-09-12T14:31:54.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/2/keyring 2020-09-12T14:31:54.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:31:54.169 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd2 key to auth repository 2020-09-12T14:31:54.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd2 key to auth repository 2020-09-12T14:31:54.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:31:54.781 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.2 2020-09-12T14:31:54.782 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.2 2020-09-12T14:31:54.782 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:31:54.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 2 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/2 --osd-journal=td/osd-backfill-space/2/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:31:54.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:31:54.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:31:54.807 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:54.807+0000 7f28360f3ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:31:54.809 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:54.808+0000 7f28360f3ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:31:54.810 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:54.809+0000 7f28360f3ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:31:55.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 2 2020-09-12T14:31:55.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:31:55.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T14:31:55.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:31:55.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:31:55.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:31:55.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:31:55.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:31:55.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:31:55.194 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:31:55.326 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:55.326+0000 7f28360f3ec0 -1 Falling back to public interface 2020-09-12T14:31:55.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:31:56.145 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:31:56.144+0000 7f28360f3ec0 -1 osd.2 0 log_to_monitors {default=true} 2020-09-12T14:31:56.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:31:56.604 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:31:56.604 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:31:56.604 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:31:56.604 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:31:56.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:31:57.020 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/181334,v1:127.0.0.1:6819/181334] [v2:127.0.0.1:6820/181334,v1:127.0.0.1:6821/181334] exists,up bddb3624-216d-4188-90f9-94cad5cb9bd8 2020-09-12T14:31:57.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:31:57.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:31:57.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:31:57.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:443: TEST_backfill_multi_partial: ceph osd set-backfillfull-ratio .85 2020-09-12T14:31:57.696 INFO:tasks.workunit.client.0.smithi027.stderr:osd set-backfillfull-ratio 0.85 2020-09-12T14:31:57.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:445: TEST_backfill_multi_partial: ceph osd set-require-min-compat-client luminous 2020-09-12T14:31:58.440 INFO:tasks.workunit.client.0.smithi027.stderr:set require_min_compat_client to luminous 2020-09-12T14:31:58.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:446: TEST_backfill_multi_partial: create_pool fillpool 1 1 2020-09-12T14:31:58.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create fillpool 1 1 2020-09-12T14:31:59.048 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'fillpool' already exists 2020-09-12T14:31:59.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:32:00.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:447: TEST_backfill_multi_partial: ceph osd pool set fillpool size 1 --yes-i-really-mean-it 2020-09-12T14:32:00.991 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 1 size to 1 2020-09-12T14:32:01.001 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:448: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:32:01.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:448: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:32:01.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:450: TEST_backfill_multi_partial: create_pool test1 1 1 2020-09-12T14:32:01.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test1 1 1 2020-09-12T14:32:01.823 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test1' already exists 2020-09-12T14:32:01.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:32:02.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:451: TEST_backfill_multi_partial: ceph osd pool set test1 size 2 2020-09-12T14:32:03.565 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 2 size to 2 2020-09-12T14:32:03.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:448: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:32:03.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:450: TEST_backfill_multi_partial: create_pool test2 1 1 2020-09-12T14:32:03.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test2 1 1 2020-09-12T14:32:04.167 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test2' already exists 2020-09-12T14:32:04.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:32:05.178 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:451: TEST_backfill_multi_partial: ceph osd pool set test2 size 2 2020-09-12T14:32:05.941 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 3 size to 2 2020-09-12T14:32:05.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:454: TEST_backfill_multi_partial: wait_for_clean 2020-09-12T14:32:05.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:32:05.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:32:05.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:32:05.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:32:05.951 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 90 .1 2020-09-12T14:32:05.952 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:32:05.952 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:32:05.952 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:32:05.952 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:32:05.952 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:32:06.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:32:06.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:32:06.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:32:06.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:32:06.047 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:32:06.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:32:06.477 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:32:06.477 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T14:32:06.477 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:32:06.477 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:32:06.477 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:32:06.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836487 2020-09-12T14:32:06.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836487 2020-09-12T14:32:06.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836487' 2020-09-12T14:32:06.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:32:06.635 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:32:06.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672967 2020-09-12T14:32:06.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672967 2020-09-12T14:32:06.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836487 1-42949672967' 2020-09-12T14:32:06.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:32:06.778 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:32:06.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509444 2020-09-12T14:32:06.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509444 2020-09-12T14:32:06.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836487 1-42949672967 2-64424509444' 2020-09-12T14:32:06.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:32:06.923 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836487 2020-09-12T14:32:06.923 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:32:06.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:32:06.923 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836487 2020-09-12T14:32:06.924 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:32:06.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836487 2020-09-12T14:32:06.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2020-09-12T14:32:06.924 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:32:06.925 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836487 2020-09-12T14:32:07.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836485 -lt 21474836487 2020-09-12T14:32:07.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:32:08.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:32:08.334 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:32:08.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836488 -lt 21474836487 2020-09-12T14:32:08.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:32:08.755 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672967 2020-09-12T14:32:08.755 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:32:08.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:32:08.756 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672967 2020-09-12T14:32:08.756 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:32:08.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672967 2020-09-12T14:32:08.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672967' 2020-09-12T14:32:08.757 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672967 2020-09-12T14:32:08.757 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:32:09.164 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672967 -lt 42949672967 2020-09-12T14:32:09.165 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:32:09.165 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509444 2020-09-12T14:32:09.165 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:32:09.165 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:32:09.166 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509444 2020-09-12T14:32:09.166 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:32:09.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509444 2020-09-12T14:32:09.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509444' 2020-09-12T14:32:09.167 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:32:09.167 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509444 2020-09-12T14:32:09.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509445 -lt 64424509444 2020-09-12T14:32:09.597 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:32:09.597 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:32:09.598 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:32:10.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 3 == 0 2020-09-12T14:32:10.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:32:10.146 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:32:10.147 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:32:10.147 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:32:10.147 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:32:10.147 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:32:10.148 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:32:10.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=3 2020-09-12T14:32:10.543 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:32:10.544 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:32:10.544 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:32:11.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 3 = 3 2020-09-12T14:32:11.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T14:32:11.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T14:32:11.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:461: TEST_backfill_multi_partial: dd if=/dev/urandom of=td/osd-backfill-space/datafile bs=2611 count=1 2020-09-12T14:32:11.078 INFO:tasks.workunit.client.0.smithi027.stderr:1+0 records in 2020-09-12T14:32:11.078 INFO:tasks.workunit.client.0.smithi027.stderr:1+0 records out 2020-09-12T14:32:11.078 INFO:tasks.workunit.client.0.smithi027.stderr:2611 bytes (2.6 kB, 2.5 KiB) copied, 5.8138e-05 s, 44.9 MB/s 2020-09-12T14:32:11.079 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: seq 1 600 2020-09-12T14:32:11.079 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:11.079 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-1 td/osd-backfill-space/datafile 2020-09-12T14:32:11.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:11.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-2 td/osd-backfill-space/datafile 2020-09-12T14:32:11.264 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:11.264 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-3 td/osd-backfill-space/datafile 2020-09-12T14:32:11.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:11.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-4 td/osd-backfill-space/datafile 2020-09-12T14:32:11.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:11.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-5 td/osd-backfill-space/datafile 2020-09-12T14:32:11.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:11.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-6 td/osd-backfill-space/datafile 2020-09-12T14:32:11.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:11.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-7 td/osd-backfill-space/datafile 2020-09-12T14:32:11.666 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:11.667 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-8 td/osd-backfill-space/datafile 2020-09-12T14:32:11.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:11.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-9 td/osd-backfill-space/datafile 2020-09-12T14:32:11.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:11.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-10 td/osd-backfill-space/datafile 2020-09-12T14:32:11.892 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:11.892 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-11 td/osd-backfill-space/datafile 2020-09-12T14:32:11.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:11.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-12 td/osd-backfill-space/datafile 2020-09-12T14:32:12.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:12.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-13 td/osd-backfill-space/datafile 2020-09-12T14:32:12.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:12.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-14 td/osd-backfill-space/datafile 2020-09-12T14:32:12.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:12.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-15 td/osd-backfill-space/datafile 2020-09-12T14:32:12.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:12.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-16 td/osd-backfill-space/datafile 2020-09-12T14:32:12.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:12.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-17 td/osd-backfill-space/datafile 2020-09-12T14:32:12.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:12.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-18 td/osd-backfill-space/datafile 2020-09-12T14:32:12.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:12.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-19 td/osd-backfill-space/datafile 2020-09-12T14:32:12.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:12.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-20 td/osd-backfill-space/datafile 2020-09-12T14:32:12.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:12.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-21 td/osd-backfill-space/datafile 2020-09-12T14:32:12.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:12.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-22 td/osd-backfill-space/datafile 2020-09-12T14:32:12.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:12.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-23 td/osd-backfill-space/datafile 2020-09-12T14:32:12.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:12.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-24 td/osd-backfill-space/datafile 2020-09-12T14:32:13.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:13.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-25 td/osd-backfill-space/datafile 2020-09-12T14:32:13.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:13.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-26 td/osd-backfill-space/datafile 2020-09-12T14:32:13.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:13.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-27 td/osd-backfill-space/datafile 2020-09-12T14:32:13.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:13.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-28 td/osd-backfill-space/datafile 2020-09-12T14:32:13.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:13.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-29 td/osd-backfill-space/datafile 2020-09-12T14:32:13.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:13.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-30 td/osd-backfill-space/datafile 2020-09-12T14:32:13.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:13.508 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-31 td/osd-backfill-space/datafile 2020-09-12T14:32:13.583 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:13.583 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-32 td/osd-backfill-space/datafile 2020-09-12T14:32:13.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:13.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-33 td/osd-backfill-space/datafile 2020-09-12T14:32:13.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:13.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-34 td/osd-backfill-space/datafile 2020-09-12T14:32:13.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:13.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-35 td/osd-backfill-space/datafile 2020-09-12T14:32:13.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:13.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-36 td/osd-backfill-space/datafile 2020-09-12T14:32:13.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:13.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-37 td/osd-backfill-space/datafile 2020-09-12T14:32:14.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:14.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-38 td/osd-backfill-space/datafile 2020-09-12T14:32:14.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:14.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-39 td/osd-backfill-space/datafile 2020-09-12T14:32:14.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:14.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-40 td/osd-backfill-space/datafile 2020-09-12T14:32:14.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:14.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-41 td/osd-backfill-space/datafile 2020-09-12T14:32:14.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:14.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-42 td/osd-backfill-space/datafile 2020-09-12T14:32:14.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:14.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-43 td/osd-backfill-space/datafile 2020-09-12T14:32:14.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:14.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-44 td/osd-backfill-space/datafile 2020-09-12T14:32:14.663 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:14.663 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-45 td/osd-backfill-space/datafile 2020-09-12T14:32:14.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:14.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-46 td/osd-backfill-space/datafile 2020-09-12T14:32:14.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:14.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-47 td/osd-backfill-space/datafile 2020-09-12T14:32:14.914 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:14.914 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-48 td/osd-backfill-space/datafile 2020-09-12T14:32:14.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:14.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-49 td/osd-backfill-space/datafile 2020-09-12T14:32:15.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:15.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-50 td/osd-backfill-space/datafile 2020-09-12T14:32:15.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:15.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-51 td/osd-backfill-space/datafile 2020-09-12T14:32:15.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:15.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-52 td/osd-backfill-space/datafile 2020-09-12T14:32:15.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:15.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-53 td/osd-backfill-space/datafile 2020-09-12T14:32:15.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:15.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-54 td/osd-backfill-space/datafile 2020-09-12T14:32:15.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:15.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-55 td/osd-backfill-space/datafile 2020-09-12T14:32:15.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:15.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-56 td/osd-backfill-space/datafile 2020-09-12T14:32:15.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:15.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-57 td/osd-backfill-space/datafile 2020-09-12T14:32:15.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:15.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-58 td/osd-backfill-space/datafile 2020-09-12T14:32:15.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:15.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-59 td/osd-backfill-space/datafile 2020-09-12T14:32:15.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:15.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-60 td/osd-backfill-space/datafile 2020-09-12T14:32:15.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:15.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-61 td/osd-backfill-space/datafile 2020-09-12T14:32:16.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:16.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-62 td/osd-backfill-space/datafile 2020-09-12T14:32:16.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:16.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-63 td/osd-backfill-space/datafile 2020-09-12T14:32:16.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:16.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-64 td/osd-backfill-space/datafile 2020-09-12T14:32:16.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:16.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-65 td/osd-backfill-space/datafile 2020-09-12T14:32:16.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:16.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-66 td/osd-backfill-space/datafile 2020-09-12T14:32:16.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:16.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-67 td/osd-backfill-space/datafile 2020-09-12T14:32:16.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:16.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-68 td/osd-backfill-space/datafile 2020-09-12T14:32:16.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:16.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-69 td/osd-backfill-space/datafile 2020-09-12T14:32:16.672 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:16.672 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-70 td/osd-backfill-space/datafile 2020-09-12T14:32:16.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:16.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-71 td/osd-backfill-space/datafile 2020-09-12T14:32:16.823 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:16.823 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-72 td/osd-backfill-space/datafile 2020-09-12T14:32:16.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:16.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-73 td/osd-backfill-space/datafile 2020-09-12T14:32:16.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:16.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-74 td/osd-backfill-space/datafile 2020-09-12T14:32:17.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:17.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-75 td/osd-backfill-space/datafile 2020-09-12T14:32:17.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:17.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-76 td/osd-backfill-space/datafile 2020-09-12T14:32:17.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:17.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-77 td/osd-backfill-space/datafile 2020-09-12T14:32:17.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:17.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-78 td/osd-backfill-space/datafile 2020-09-12T14:32:17.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:17.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-79 td/osd-backfill-space/datafile 2020-09-12T14:32:17.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:17.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-80 td/osd-backfill-space/datafile 2020-09-12T14:32:17.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:17.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-81 td/osd-backfill-space/datafile 2020-09-12T14:32:17.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:17.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-82 td/osd-backfill-space/datafile 2020-09-12T14:32:17.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:17.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-83 td/osd-backfill-space/datafile 2020-09-12T14:32:17.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:17.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-84 td/osd-backfill-space/datafile 2020-09-12T14:32:17.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:17.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-85 td/osd-backfill-space/datafile 2020-09-12T14:32:17.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:17.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-86 td/osd-backfill-space/datafile 2020-09-12T14:32:17.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:17.953 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-87 td/osd-backfill-space/datafile 2020-09-12T14:32:18.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:18.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-88 td/osd-backfill-space/datafile 2020-09-12T14:32:18.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:18.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-89 td/osd-backfill-space/datafile 2020-09-12T14:32:18.178 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:18.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-90 td/osd-backfill-space/datafile 2020-09-12T14:32:18.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:18.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-91 td/osd-backfill-space/datafile 2020-09-12T14:32:18.354 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:18.354 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-92 td/osd-backfill-space/datafile 2020-09-12T14:32:18.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:18.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-93 td/osd-backfill-space/datafile 2020-09-12T14:32:18.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:18.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-94 td/osd-backfill-space/datafile 2020-09-12T14:32:18.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:18.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-95 td/osd-backfill-space/datafile 2020-09-12T14:32:18.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:18.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-96 td/osd-backfill-space/datafile 2020-09-12T14:32:18.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:18.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-97 td/osd-backfill-space/datafile 2020-09-12T14:32:18.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:18.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-98 td/osd-backfill-space/datafile 2020-09-12T14:32:18.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:18.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-99 td/osd-backfill-space/datafile 2020-09-12T14:32:18.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:18.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-100 td/osd-backfill-space/datafile 2020-09-12T14:32:19.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:19.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-101 td/osd-backfill-space/datafile 2020-09-12T14:32:19.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:19.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-102 td/osd-backfill-space/datafile 2020-09-12T14:32:19.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:19.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-103 td/osd-backfill-space/datafile 2020-09-12T14:32:19.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:19.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-104 td/osd-backfill-space/datafile 2020-09-12T14:32:19.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:19.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-105 td/osd-backfill-space/datafile 2020-09-12T14:32:19.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:19.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-106 td/osd-backfill-space/datafile 2020-09-12T14:32:19.551 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:19.552 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-107 td/osd-backfill-space/datafile 2020-09-12T14:32:19.626 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:19.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-108 td/osd-backfill-space/datafile 2020-09-12T14:32:19.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:19.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-109 td/osd-backfill-space/datafile 2020-09-12T14:32:19.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:19.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-110 td/osd-backfill-space/datafile 2020-09-12T14:32:19.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:19.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-111 td/osd-backfill-space/datafile 2020-09-12T14:32:19.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:19.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-112 td/osd-backfill-space/datafile 2020-09-12T14:32:20.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:20.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-113 td/osd-backfill-space/datafile 2020-09-12T14:32:20.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:20.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-114 td/osd-backfill-space/datafile 2020-09-12T14:32:20.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:20.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-115 td/osd-backfill-space/datafile 2020-09-12T14:32:20.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:20.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-116 td/osd-backfill-space/datafile 2020-09-12T14:32:20.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:20.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-117 td/osd-backfill-space/datafile 2020-09-12T14:32:20.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:20.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-118 td/osd-backfill-space/datafile 2020-09-12T14:32:20.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:20.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-119 td/osd-backfill-space/datafile 2020-09-12T14:32:20.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:20.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-120 td/osd-backfill-space/datafile 2020-09-12T14:32:20.673 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:20.673 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-121 td/osd-backfill-space/datafile 2020-09-12T14:32:20.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:20.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-122 td/osd-backfill-space/datafile 2020-09-12T14:32:20.823 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:20.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-123 td/osd-backfill-space/datafile 2020-09-12T14:32:20.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:20.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-124 td/osd-backfill-space/datafile 2020-09-12T14:32:20.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:20.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-125 td/osd-backfill-space/datafile 2020-09-12T14:32:21.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:21.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-126 td/osd-backfill-space/datafile 2020-09-12T14:32:21.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:21.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-127 td/osd-backfill-space/datafile 2020-09-12T14:32:21.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:21.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-128 td/osd-backfill-space/datafile 2020-09-12T14:32:21.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:21.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-129 td/osd-backfill-space/datafile 2020-09-12T14:32:21.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:21.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-130 td/osd-backfill-space/datafile 2020-09-12T14:32:21.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:21.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-131 td/osd-backfill-space/datafile 2020-09-12T14:32:21.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:21.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-132 td/osd-backfill-space/datafile 2020-09-12T14:32:21.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:21.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-133 td/osd-backfill-space/datafile 2020-09-12T14:32:21.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:21.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-134 td/osd-backfill-space/datafile 2020-09-12T14:32:21.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:21.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-135 td/osd-backfill-space/datafile 2020-09-12T14:32:21.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:21.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-136 td/osd-backfill-space/datafile 2020-09-12T14:32:21.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:21.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-137 td/osd-backfill-space/datafile 2020-09-12T14:32:21.953 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:21.953 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-138 td/osd-backfill-space/datafile 2020-09-12T14:32:22.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:22.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-139 td/osd-backfill-space/datafile 2020-09-12T14:32:22.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:22.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-140 td/osd-backfill-space/datafile 2020-09-12T14:32:22.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:22.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-141 td/osd-backfill-space/datafile 2020-09-12T14:32:22.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:22.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-142 td/osd-backfill-space/datafile 2020-09-12T14:32:22.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:22.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-143 td/osd-backfill-space/datafile 2020-09-12T14:32:22.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:22.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-144 td/osd-backfill-space/datafile 2020-09-12T14:32:22.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:22.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-145 td/osd-backfill-space/datafile 2020-09-12T14:32:22.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:22.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-146 td/osd-backfill-space/datafile 2020-09-12T14:32:22.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:22.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-147 td/osd-backfill-space/datafile 2020-09-12T14:32:22.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:22.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-148 td/osd-backfill-space/datafile 2020-09-12T14:32:22.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:22.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-149 td/osd-backfill-space/datafile 2020-09-12T14:32:22.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:22.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-150 td/osd-backfill-space/datafile 2020-09-12T14:32:23.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:23.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-151 td/osd-backfill-space/datafile 2020-09-12T14:32:23.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:23.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-152 td/osd-backfill-space/datafile 2020-09-12T14:32:23.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:23.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-153 td/osd-backfill-space/datafile 2020-09-12T14:32:23.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:23.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-154 td/osd-backfill-space/datafile 2020-09-12T14:32:23.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:23.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-155 td/osd-backfill-space/datafile 2020-09-12T14:32:23.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:23.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-156 td/osd-backfill-space/datafile 2020-09-12T14:32:23.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:23.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-157 td/osd-backfill-space/datafile 2020-09-12T14:32:23.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:23.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-158 td/osd-backfill-space/datafile 2020-09-12T14:32:23.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:23.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-159 td/osd-backfill-space/datafile 2020-09-12T14:32:23.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:23.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-160 td/osd-backfill-space/datafile 2020-09-12T14:32:23.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:23.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-161 td/osd-backfill-space/datafile 2020-09-12T14:32:23.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:23.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-162 td/osd-backfill-space/datafile 2020-09-12T14:32:23.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:23.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-163 td/osd-backfill-space/datafile 2020-09-12T14:32:24.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:24.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-164 td/osd-backfill-space/datafile 2020-09-12T14:32:24.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:24.113 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-165 td/osd-backfill-space/datafile 2020-09-12T14:32:24.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:24.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-166 td/osd-backfill-space/datafile 2020-09-12T14:32:24.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:24.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-167 td/osd-backfill-space/datafile 2020-09-12T14:32:24.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:24.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-168 td/osd-backfill-space/datafile 2020-09-12T14:32:24.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:24.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-169 td/osd-backfill-space/datafile 2020-09-12T14:32:24.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:24.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-170 td/osd-backfill-space/datafile 2020-09-12T14:32:24.665 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:24.665 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-171 td/osd-backfill-space/datafile 2020-09-12T14:32:24.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:24.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-172 td/osd-backfill-space/datafile 2020-09-12T14:32:24.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:24.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-173 td/osd-backfill-space/datafile 2020-09-12T14:32:24.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:24.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-174 td/osd-backfill-space/datafile 2020-09-12T14:32:25.066 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:25.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-175 td/osd-backfill-space/datafile 2020-09-12T14:32:25.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:25.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-176 td/osd-backfill-space/datafile 2020-09-12T14:32:25.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:25.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-177 td/osd-backfill-space/datafile 2020-09-12T14:32:25.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:25.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-178 td/osd-backfill-space/datafile 2020-09-12T14:32:25.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:25.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-179 td/osd-backfill-space/datafile 2020-09-12T14:32:25.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:25.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-180 td/osd-backfill-space/datafile 2020-09-12T14:32:25.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:25.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-181 td/osd-backfill-space/datafile 2020-09-12T14:32:25.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:25.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-182 td/osd-backfill-space/datafile 2020-09-12T14:32:25.663 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:25.663 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-183 td/osd-backfill-space/datafile 2020-09-12T14:32:25.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:25.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-184 td/osd-backfill-space/datafile 2020-09-12T14:32:25.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:25.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-185 td/osd-backfill-space/datafile 2020-09-12T14:32:25.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:25.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-186 td/osd-backfill-space/datafile 2020-09-12T14:32:25.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:25.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-187 td/osd-backfill-space/datafile 2020-09-12T14:32:26.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:26.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-188 td/osd-backfill-space/datafile 2020-09-12T14:32:26.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:26.116 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-189 td/osd-backfill-space/datafile 2020-09-12T14:32:26.191 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:26.191 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-190 td/osd-backfill-space/datafile 2020-09-12T14:32:26.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:26.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-191 td/osd-backfill-space/datafile 2020-09-12T14:32:26.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:26.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-192 td/osd-backfill-space/datafile 2020-09-12T14:32:26.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:26.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-193 td/osd-backfill-space/datafile 2020-09-12T14:32:26.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:26.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-194 td/osd-backfill-space/datafile 2020-09-12T14:32:26.668 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:26.668 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-195 td/osd-backfill-space/datafile 2020-09-12T14:32:26.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:26.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-196 td/osd-backfill-space/datafile 2020-09-12T14:32:26.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:26.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-197 td/osd-backfill-space/datafile 2020-09-12T14:32:26.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:26.894 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-198 td/osd-backfill-space/datafile 2020-09-12T14:32:26.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:26.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-199 td/osd-backfill-space/datafile 2020-09-12T14:32:27.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:27.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-200 td/osd-backfill-space/datafile 2020-09-12T14:32:27.119 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:27.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-201 td/osd-backfill-space/datafile 2020-09-12T14:32:27.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:27.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-202 td/osd-backfill-space/datafile 2020-09-12T14:32:27.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:27.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-203 td/osd-backfill-space/datafile 2020-09-12T14:32:27.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:27.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-204 td/osd-backfill-space/datafile 2020-09-12T14:32:27.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:27.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-205 td/osd-backfill-space/datafile 2020-09-12T14:32:27.496 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:27.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-206 td/osd-backfill-space/datafile 2020-09-12T14:32:27.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:27.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-207 td/osd-backfill-space/datafile 2020-09-12T14:32:27.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:27.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-208 td/osd-backfill-space/datafile 2020-09-12T14:32:27.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:27.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-209 td/osd-backfill-space/datafile 2020-09-12T14:32:27.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:27.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-210 td/osd-backfill-space/datafile 2020-09-12T14:32:27.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:27.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-211 td/osd-backfill-space/datafile 2020-09-12T14:32:27.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:27.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-212 td/osd-backfill-space/datafile 2020-09-12T14:32:28.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:28.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-213 td/osd-backfill-space/datafile 2020-09-12T14:32:28.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:28.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-214 td/osd-backfill-space/datafile 2020-09-12T14:32:28.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:28.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-215 td/osd-backfill-space/datafile 2020-09-12T14:32:28.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:28.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-216 td/osd-backfill-space/datafile 2020-09-12T14:32:28.366 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:28.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-217 td/osd-backfill-space/datafile 2020-09-12T14:32:28.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:28.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-218 td/osd-backfill-space/datafile 2020-09-12T14:32:28.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:28.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-219 td/osd-backfill-space/datafile 2020-09-12T14:32:28.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:28.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-220 td/osd-backfill-space/datafile 2020-09-12T14:32:28.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:28.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-221 td/osd-backfill-space/datafile 2020-09-12T14:32:28.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:28.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-222 td/osd-backfill-space/datafile 2020-09-12T14:32:28.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:28.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-223 td/osd-backfill-space/datafile 2020-09-12T14:32:28.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:28.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-224 td/osd-backfill-space/datafile 2020-09-12T14:32:29.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:29.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-225 td/osd-backfill-space/datafile 2020-09-12T14:32:29.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:29.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-226 td/osd-backfill-space/datafile 2020-09-12T14:32:29.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:29.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-227 td/osd-backfill-space/datafile 2020-09-12T14:32:29.262 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:29.262 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-228 td/osd-backfill-space/datafile 2020-09-12T14:32:29.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:29.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-229 td/osd-backfill-space/datafile 2020-09-12T14:32:29.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:29.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-230 td/osd-backfill-space/datafile 2020-09-12T14:32:29.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:29.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-231 td/osd-backfill-space/datafile 2020-09-12T14:32:29.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:29.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-232 td/osd-backfill-space/datafile 2020-09-12T14:32:29.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:29.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-233 td/osd-backfill-space/datafile 2020-09-12T14:32:29.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:29.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-234 td/osd-backfill-space/datafile 2020-09-12T14:32:29.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:29.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-235 td/osd-backfill-space/datafile 2020-09-12T14:32:29.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:29.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-236 td/osd-backfill-space/datafile 2020-09-12T14:32:29.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:29.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-237 td/osd-backfill-space/datafile 2020-09-12T14:32:30.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:30.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-238 td/osd-backfill-space/datafile 2020-09-12T14:32:30.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:30.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-239 td/osd-backfill-space/datafile 2020-09-12T14:32:30.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:30.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-240 td/osd-backfill-space/datafile 2020-09-12T14:32:30.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:30.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-241 td/osd-backfill-space/datafile 2020-09-12T14:32:30.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:30.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-242 td/osd-backfill-space/datafile 2020-09-12T14:32:30.433 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:30.433 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-243 td/osd-backfill-space/datafile 2020-09-12T14:32:30.508 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:30.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-244 td/osd-backfill-space/datafile 2020-09-12T14:32:30.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:30.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-245 td/osd-backfill-space/datafile 2020-09-12T14:32:30.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:30.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-246 td/osd-backfill-space/datafile 2020-09-12T14:32:30.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:30.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-247 td/osd-backfill-space/datafile 2020-09-12T14:32:30.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:30.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-248 td/osd-backfill-space/datafile 2020-09-12T14:32:30.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:30.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-249 td/osd-backfill-space/datafile 2020-09-12T14:32:30.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:30.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-250 td/osd-backfill-space/datafile 2020-09-12T14:32:31.086 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:31.086 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-251 td/osd-backfill-space/datafile 2020-09-12T14:32:31.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:31.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-252 td/osd-backfill-space/datafile 2020-09-12T14:32:31.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:31.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-253 td/osd-backfill-space/datafile 2020-09-12T14:32:31.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:31.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-254 td/osd-backfill-space/datafile 2020-09-12T14:32:31.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:31.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-255 td/osd-backfill-space/datafile 2020-09-12T14:32:31.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:31.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-256 td/osd-backfill-space/datafile 2020-09-12T14:32:31.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:31.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-257 td/osd-backfill-space/datafile 2020-09-12T14:32:31.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:31.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-258 td/osd-backfill-space/datafile 2020-09-12T14:32:31.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:31.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-259 td/osd-backfill-space/datafile 2020-09-12T14:32:31.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:31.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-260 td/osd-backfill-space/datafile 2020-09-12T14:32:31.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:31.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-261 td/osd-backfill-space/datafile 2020-09-12T14:32:31.915 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:31.915 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-262 td/osd-backfill-space/datafile 2020-09-12T14:32:31.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:31.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-263 td/osd-backfill-space/datafile 2020-09-12T14:32:32.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:32.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-264 td/osd-backfill-space/datafile 2020-09-12T14:32:32.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:32.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-265 td/osd-backfill-space/datafile 2020-09-12T14:32:32.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:32.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-266 td/osd-backfill-space/datafile 2020-09-12T14:32:32.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:32.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-267 td/osd-backfill-space/datafile 2020-09-12T14:32:32.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:32.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-268 td/osd-backfill-space/datafile 2020-09-12T14:32:32.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:32.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-269 td/osd-backfill-space/datafile 2020-09-12T14:32:32.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:32.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-270 td/osd-backfill-space/datafile 2020-09-12T14:32:32.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:32.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-271 td/osd-backfill-space/datafile 2020-09-12T14:32:32.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:32.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-272 td/osd-backfill-space/datafile 2020-09-12T14:32:32.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:32.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-273 td/osd-backfill-space/datafile 2020-09-12T14:32:32.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:32.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-274 td/osd-backfill-space/datafile 2020-09-12T14:32:32.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:32.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-275 td/osd-backfill-space/datafile 2020-09-12T14:32:33.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:33.062 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-276 td/osd-backfill-space/datafile 2020-09-12T14:32:33.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:33.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-277 td/osd-backfill-space/datafile 2020-09-12T14:32:33.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:33.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-278 td/osd-backfill-space/datafile 2020-09-12T14:32:33.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:33.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-279 td/osd-backfill-space/datafile 2020-09-12T14:32:33.371 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:33.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-280 td/osd-backfill-space/datafile 2020-09-12T14:32:33.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:33.447 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-281 td/osd-backfill-space/datafile 2020-09-12T14:32:33.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:33.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-282 td/osd-backfill-space/datafile 2020-09-12T14:32:33.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:33.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-283 td/osd-backfill-space/datafile 2020-09-12T14:32:33.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:33.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-284 td/osd-backfill-space/datafile 2020-09-12T14:32:33.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:33.765 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-285 td/osd-backfill-space/datafile 2020-09-12T14:32:33.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:33.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-286 td/osd-backfill-space/datafile 2020-09-12T14:32:33.915 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:33.915 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-287 td/osd-backfill-space/datafile 2020-09-12T14:32:33.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:33.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-288 td/osd-backfill-space/datafile 2020-09-12T14:32:34.066 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:34.066 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-289 td/osd-backfill-space/datafile 2020-09-12T14:32:34.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:34.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-290 td/osd-backfill-space/datafile 2020-09-12T14:32:34.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:34.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-291 td/osd-backfill-space/datafile 2020-09-12T14:32:34.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:34.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-292 td/osd-backfill-space/datafile 2020-09-12T14:32:34.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:34.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-293 td/osd-backfill-space/datafile 2020-09-12T14:32:34.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:34.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-294 td/osd-backfill-space/datafile 2020-09-12T14:32:34.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:34.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-295 td/osd-backfill-space/datafile 2020-09-12T14:32:34.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:34.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-296 td/osd-backfill-space/datafile 2020-09-12T14:32:34.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:34.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-297 td/osd-backfill-space/datafile 2020-09-12T14:32:34.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:34.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-298 td/osd-backfill-space/datafile 2020-09-12T14:32:34.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:34.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-299 td/osd-backfill-space/datafile 2020-09-12T14:32:35.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:35.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-300 td/osd-backfill-space/datafile 2020-09-12T14:32:35.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:35.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-301 td/osd-backfill-space/datafile 2020-09-12T14:32:35.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:35.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-302 td/osd-backfill-space/datafile 2020-09-12T14:32:35.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:35.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-303 td/osd-backfill-space/datafile 2020-09-12T14:32:35.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:35.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-304 td/osd-backfill-space/datafile 2020-09-12T14:32:35.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:35.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-305 td/osd-backfill-space/datafile 2020-09-12T14:32:35.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:35.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-306 td/osd-backfill-space/datafile 2020-09-12T14:32:35.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:35.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-307 td/osd-backfill-space/datafile 2020-09-12T14:32:35.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:35.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-308 td/osd-backfill-space/datafile 2020-09-12T14:32:36.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:36.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-309 td/osd-backfill-space/datafile 2020-09-12T14:32:36.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:36.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-310 td/osd-backfill-space/datafile 2020-09-12T14:32:36.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:36.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-311 td/osd-backfill-space/datafile 2020-09-12T14:32:36.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:36.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-312 td/osd-backfill-space/datafile 2020-09-12T14:32:36.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:36.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-313 td/osd-backfill-space/datafile 2020-09-12T14:32:36.466 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:36.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-314 td/osd-backfill-space/datafile 2020-09-12T14:32:36.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:36.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-315 td/osd-backfill-space/datafile 2020-09-12T14:32:36.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:36.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-316 td/osd-backfill-space/datafile 2020-09-12T14:32:36.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:36.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-317 td/osd-backfill-space/datafile 2020-09-12T14:32:36.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:36.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-318 td/osd-backfill-space/datafile 2020-09-12T14:32:36.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:36.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-319 td/osd-backfill-space/datafile 2020-09-12T14:32:36.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:36.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-320 td/osd-backfill-space/datafile 2020-09-12T14:32:37.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:37.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-321 td/osd-backfill-space/datafile 2020-09-12T14:32:37.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:37.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-322 td/osd-backfill-space/datafile 2020-09-12T14:32:37.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:37.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-323 td/osd-backfill-space/datafile 2020-09-12T14:32:37.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:37.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-324 td/osd-backfill-space/datafile 2020-09-12T14:32:37.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:37.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-325 td/osd-backfill-space/datafile 2020-09-12T14:32:37.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:37.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-326 td/osd-backfill-space/datafile 2020-09-12T14:32:37.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:37.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-327 td/osd-backfill-space/datafile 2020-09-12T14:32:37.606 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:37.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-328 td/osd-backfill-space/datafile 2020-09-12T14:32:37.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:37.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-329 td/osd-backfill-space/datafile 2020-09-12T14:32:37.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:37.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-330 td/osd-backfill-space/datafile 2020-09-12T14:32:37.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:37.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-331 td/osd-backfill-space/datafile 2020-09-12T14:32:37.908 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:37.908 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-332 td/osd-backfill-space/datafile 2020-09-12T14:32:37.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:37.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-333 td/osd-backfill-space/datafile 2020-09-12T14:32:38.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:38.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-334 td/osd-backfill-space/datafile 2020-09-12T14:32:38.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:38.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-335 td/osd-backfill-space/datafile 2020-09-12T14:32:38.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:38.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-336 td/osd-backfill-space/datafile 2020-09-12T14:32:38.284 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:38.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-337 td/osd-backfill-space/datafile 2020-09-12T14:32:38.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:38.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-338 td/osd-backfill-space/datafile 2020-09-12T14:32:38.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:38.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-339 td/osd-backfill-space/datafile 2020-09-12T14:32:38.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:38.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-340 td/osd-backfill-space/datafile 2020-09-12T14:32:38.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:38.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-341 td/osd-backfill-space/datafile 2020-09-12T14:32:38.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:38.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-342 td/osd-backfill-space/datafile 2020-09-12T14:32:38.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:38.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-343 td/osd-backfill-space/datafile 2020-09-12T14:32:38.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:38.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-344 td/osd-backfill-space/datafile 2020-09-12T14:32:38.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:38.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-345 td/osd-backfill-space/datafile 2020-09-12T14:32:39.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:39.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-346 td/osd-backfill-space/datafile 2020-09-12T14:32:39.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:39.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-347 td/osd-backfill-space/datafile 2020-09-12T14:32:39.222 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:39.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-348 td/osd-backfill-space/datafile 2020-09-12T14:32:39.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:39.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-349 td/osd-backfill-space/datafile 2020-09-12T14:32:39.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:39.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-350 td/osd-backfill-space/datafile 2020-09-12T14:32:39.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:39.457 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-351 td/osd-backfill-space/datafile 2020-09-12T14:32:39.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:39.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-352 td/osd-backfill-space/datafile 2020-09-12T14:32:39.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:39.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-353 td/osd-backfill-space/datafile 2020-09-12T14:32:39.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:39.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-354 td/osd-backfill-space/datafile 2020-09-12T14:32:39.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:39.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-355 td/osd-backfill-space/datafile 2020-09-12T14:32:39.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:39.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-356 td/osd-backfill-space/datafile 2020-09-12T14:32:39.908 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:39.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-357 td/osd-backfill-space/datafile 2020-09-12T14:32:39.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:39.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-358 td/osd-backfill-space/datafile 2020-09-12T14:32:40.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:40.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-359 td/osd-backfill-space/datafile 2020-09-12T14:32:40.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:40.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-360 td/osd-backfill-space/datafile 2020-09-12T14:32:40.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:40.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-361 td/osd-backfill-space/datafile 2020-09-12T14:32:40.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:40.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-362 td/osd-backfill-space/datafile 2020-09-12T14:32:40.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:40.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-363 td/osd-backfill-space/datafile 2020-09-12T14:32:40.477 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:40.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-364 td/osd-backfill-space/datafile 2020-09-12T14:32:40.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:40.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-365 td/osd-backfill-space/datafile 2020-09-12T14:32:40.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:40.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-366 td/osd-backfill-space/datafile 2020-09-12T14:32:40.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:40.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-367 td/osd-backfill-space/datafile 2020-09-12T14:32:40.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:40.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-368 td/osd-backfill-space/datafile 2020-09-12T14:32:40.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:40.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-369 td/osd-backfill-space/datafile 2020-09-12T14:32:41.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:41.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-370 td/osd-backfill-space/datafile 2020-09-12T14:32:41.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:41.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-371 td/osd-backfill-space/datafile 2020-09-12T14:32:41.172 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:41.172 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-372 td/osd-backfill-space/datafile 2020-09-12T14:32:41.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:41.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-373 td/osd-backfill-space/datafile 2020-09-12T14:32:41.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:41.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-374 td/osd-backfill-space/datafile 2020-09-12T14:32:41.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:41.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-375 td/osd-backfill-space/datafile 2020-09-12T14:32:41.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:41.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-376 td/osd-backfill-space/datafile 2020-09-12T14:32:41.548 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:41.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-377 td/osd-backfill-space/datafile 2020-09-12T14:32:41.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:41.625 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-378 td/osd-backfill-space/datafile 2020-09-12T14:32:41.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:41.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-379 td/osd-backfill-space/datafile 2020-09-12T14:32:41.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:41.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-380 td/osd-backfill-space/datafile 2020-09-12T14:32:41.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:41.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-381 td/osd-backfill-space/datafile 2020-09-12T14:32:41.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:41.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-382 td/osd-backfill-space/datafile 2020-09-12T14:32:42.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:42.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-383 td/osd-backfill-space/datafile 2020-09-12T14:32:42.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:42.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-384 td/osd-backfill-space/datafile 2020-09-12T14:32:42.151 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:42.151 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-385 td/osd-backfill-space/datafile 2020-09-12T14:32:42.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:42.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-386 td/osd-backfill-space/datafile 2020-09-12T14:32:42.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:42.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-387 td/osd-backfill-space/datafile 2020-09-12T14:32:42.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:42.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-388 td/osd-backfill-space/datafile 2020-09-12T14:32:42.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:42.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-389 td/osd-backfill-space/datafile 2020-09-12T14:32:42.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:42.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-390 td/osd-backfill-space/datafile 2020-09-12T14:32:42.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:42.671 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-391 td/osd-backfill-space/datafile 2020-09-12T14:32:42.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:42.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-392 td/osd-backfill-space/datafile 2020-09-12T14:32:42.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:42.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-393 td/osd-backfill-space/datafile 2020-09-12T14:32:42.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:42.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-394 td/osd-backfill-space/datafile 2020-09-12T14:32:42.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:42.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-395 td/osd-backfill-space/datafile 2020-09-12T14:32:43.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:43.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-396 td/osd-backfill-space/datafile 2020-09-12T14:32:43.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:43.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-397 td/osd-backfill-space/datafile 2020-09-12T14:32:43.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:43.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-398 td/osd-backfill-space/datafile 2020-09-12T14:32:43.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:43.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-399 td/osd-backfill-space/datafile 2020-09-12T14:32:43.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:43.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-400 td/osd-backfill-space/datafile 2020-09-12T14:32:43.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:43.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-401 td/osd-backfill-space/datafile 2020-09-12T14:32:43.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:43.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-402 td/osd-backfill-space/datafile 2020-09-12T14:32:43.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:43.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-403 td/osd-backfill-space/datafile 2020-09-12T14:32:43.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:43.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-404 td/osd-backfill-space/datafile 2020-09-12T14:32:43.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:43.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-405 td/osd-backfill-space/datafile 2020-09-12T14:32:43.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:43.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-406 td/osd-backfill-space/datafile 2020-09-12T14:32:43.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:43.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-407 td/osd-backfill-space/datafile 2020-09-12T14:32:43.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:43.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-408 td/osd-backfill-space/datafile 2020-09-12T14:32:44.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:44.035 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-409 td/osd-backfill-space/datafile 2020-09-12T14:32:44.110 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:44.110 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-410 td/osd-backfill-space/datafile 2020-09-12T14:32:44.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:44.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-411 td/osd-backfill-space/datafile 2020-09-12T14:32:44.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:44.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-412 td/osd-backfill-space/datafile 2020-09-12T14:32:44.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:44.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-413 td/osd-backfill-space/datafile 2020-09-12T14:32:44.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:44.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-414 td/osd-backfill-space/datafile 2020-09-12T14:32:44.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:44.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-415 td/osd-backfill-space/datafile 2020-09-12T14:32:44.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:44.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-416 td/osd-backfill-space/datafile 2020-09-12T14:32:44.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:44.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-417 td/osd-backfill-space/datafile 2020-09-12T14:32:44.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:44.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-418 td/osd-backfill-space/datafile 2020-09-12T14:32:44.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:44.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-419 td/osd-backfill-space/datafile 2020-09-12T14:32:44.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:44.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-420 td/osd-backfill-space/datafile 2020-09-12T14:32:45.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:45.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-421 td/osd-backfill-space/datafile 2020-09-12T14:32:45.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:45.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-422 td/osd-backfill-space/datafile 2020-09-12T14:32:45.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:45.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-423 td/osd-backfill-space/datafile 2020-09-12T14:32:45.298 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:45.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-424 td/osd-backfill-space/datafile 2020-09-12T14:32:45.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:45.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-425 td/osd-backfill-space/datafile 2020-09-12T14:32:45.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:45.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-426 td/osd-backfill-space/datafile 2020-09-12T14:32:45.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:45.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-427 td/osd-backfill-space/datafile 2020-09-12T14:32:45.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:45.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-428 td/osd-backfill-space/datafile 2020-09-12T14:32:45.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:45.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-429 td/osd-backfill-space/datafile 2020-09-12T14:32:45.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:45.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-430 td/osd-backfill-space/datafile 2020-09-12T14:32:45.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:45.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-431 td/osd-backfill-space/datafile 2020-09-12T14:32:45.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:45.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-432 td/osd-backfill-space/datafile 2020-09-12T14:32:46.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:46.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-433 td/osd-backfill-space/datafile 2020-09-12T14:32:46.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:46.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-434 td/osd-backfill-space/datafile 2020-09-12T14:32:46.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:46.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-435 td/osd-backfill-space/datafile 2020-09-12T14:32:46.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:46.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-436 td/osd-backfill-space/datafile 2020-09-12T14:32:46.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:46.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-437 td/osd-backfill-space/datafile 2020-09-12T14:32:46.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:46.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-438 td/osd-backfill-space/datafile 2020-09-12T14:32:46.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:46.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-439 td/osd-backfill-space/datafile 2020-09-12T14:32:46.662 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:46.663 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-440 td/osd-backfill-space/datafile 2020-09-12T14:32:46.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:46.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-441 td/osd-backfill-space/datafile 2020-09-12T14:32:46.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:46.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-442 td/osd-backfill-space/datafile 2020-09-12T14:32:46.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:46.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-443 td/osd-backfill-space/datafile 2020-09-12T14:32:46.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:46.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-444 td/osd-backfill-space/datafile 2020-09-12T14:32:47.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:47.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-445 td/osd-backfill-space/datafile 2020-09-12T14:32:47.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:47.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-446 td/osd-backfill-space/datafile 2020-09-12T14:32:47.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:47.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-447 td/osd-backfill-space/datafile 2020-09-12T14:32:47.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:47.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-448 td/osd-backfill-space/datafile 2020-09-12T14:32:47.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:47.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-449 td/osd-backfill-space/datafile 2020-09-12T14:32:47.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:47.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-450 td/osd-backfill-space/datafile 2020-09-12T14:32:47.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:47.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-451 td/osd-backfill-space/datafile 2020-09-12T14:32:47.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:47.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-452 td/osd-backfill-space/datafile 2020-09-12T14:32:47.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:47.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-453 td/osd-backfill-space/datafile 2020-09-12T14:32:47.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:47.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-454 td/osd-backfill-space/datafile 2020-09-12T14:32:47.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:47.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-455 td/osd-backfill-space/datafile 2020-09-12T14:32:47.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:47.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-456 td/osd-backfill-space/datafile 2020-09-12T14:32:47.943 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:47.943 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-457 td/osd-backfill-space/datafile 2020-09-12T14:32:48.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:48.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-458 td/osd-backfill-space/datafile 2020-09-12T14:32:48.093 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:48.093 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-459 td/osd-backfill-space/datafile 2020-09-12T14:32:48.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:48.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-460 td/osd-backfill-space/datafile 2020-09-12T14:32:48.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:48.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-461 td/osd-backfill-space/datafile 2020-09-12T14:32:48.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:48.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-462 td/osd-backfill-space/datafile 2020-09-12T14:32:48.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:48.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-463 td/osd-backfill-space/datafile 2020-09-12T14:32:48.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:48.496 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-464 td/osd-backfill-space/datafile 2020-09-12T14:32:48.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:48.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-465 td/osd-backfill-space/datafile 2020-09-12T14:32:48.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:48.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-466 td/osd-backfill-space/datafile 2020-09-12T14:32:48.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:48.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-467 td/osd-backfill-space/datafile 2020-09-12T14:32:48.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:48.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-468 td/osd-backfill-space/datafile 2020-09-12T14:32:48.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:48.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-469 td/osd-backfill-space/datafile 2020-09-12T14:32:48.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:48.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-470 td/osd-backfill-space/datafile 2020-09-12T14:32:49.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:49.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-471 td/osd-backfill-space/datafile 2020-09-12T14:32:49.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:49.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-472 td/osd-backfill-space/datafile 2020-09-12T14:32:49.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:49.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-473 td/osd-backfill-space/datafile 2020-09-12T14:32:49.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:49.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-474 td/osd-backfill-space/datafile 2020-09-12T14:32:49.366 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:49.366 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-475 td/osd-backfill-space/datafile 2020-09-12T14:32:49.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:49.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-476 td/osd-backfill-space/datafile 2020-09-12T14:32:49.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:49.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-477 td/osd-backfill-space/datafile 2020-09-12T14:32:49.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:49.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-478 td/osd-backfill-space/datafile 2020-09-12T14:32:49.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:49.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-479 td/osd-backfill-space/datafile 2020-09-12T14:32:49.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:49.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-480 td/osd-backfill-space/datafile 2020-09-12T14:32:49.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:49.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-481 td/osd-backfill-space/datafile 2020-09-12T14:32:49.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:49.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-482 td/osd-backfill-space/datafile 2020-09-12T14:32:49.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:49.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-483 td/osd-backfill-space/datafile 2020-09-12T14:32:50.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:50.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-484 td/osd-backfill-space/datafile 2020-09-12T14:32:50.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:50.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-485 td/osd-backfill-space/datafile 2020-09-12T14:32:50.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:50.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-486 td/osd-backfill-space/datafile 2020-09-12T14:32:50.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:50.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-487 td/osd-backfill-space/datafile 2020-09-12T14:32:50.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:50.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-488 td/osd-backfill-space/datafile 2020-09-12T14:32:50.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:50.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-489 td/osd-backfill-space/datafile 2020-09-12T14:32:50.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:50.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-490 td/osd-backfill-space/datafile 2020-09-12T14:32:50.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:50.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-491 td/osd-backfill-space/datafile 2020-09-12T14:32:50.730 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:50.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-492 td/osd-backfill-space/datafile 2020-09-12T14:32:50.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:50.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-493 td/osd-backfill-space/datafile 2020-09-12T14:32:50.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:50.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-494 td/osd-backfill-space/datafile 2020-09-12T14:32:50.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:50.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-495 td/osd-backfill-space/datafile 2020-09-12T14:32:51.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:51.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-496 td/osd-backfill-space/datafile 2020-09-12T14:32:51.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:51.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-497 td/osd-backfill-space/datafile 2020-09-12T14:32:51.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:51.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-498 td/osd-backfill-space/datafile 2020-09-12T14:32:51.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:51.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-499 td/osd-backfill-space/datafile 2020-09-12T14:32:51.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:51.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-500 td/osd-backfill-space/datafile 2020-09-12T14:32:51.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:51.417 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-501 td/osd-backfill-space/datafile 2020-09-12T14:32:51.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:51.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-502 td/osd-backfill-space/datafile 2020-09-12T14:32:51.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:51.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-503 td/osd-backfill-space/datafile 2020-09-12T14:32:51.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:51.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-504 td/osd-backfill-space/datafile 2020-09-12T14:32:51.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:51.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-505 td/osd-backfill-space/datafile 2020-09-12T14:32:51.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:51.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-506 td/osd-backfill-space/datafile 2020-09-12T14:32:51.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:51.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-507 td/osd-backfill-space/datafile 2020-09-12T14:32:51.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:51.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-508 td/osd-backfill-space/datafile 2020-09-12T14:32:52.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:52.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-509 td/osd-backfill-space/datafile 2020-09-12T14:32:52.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:52.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-510 td/osd-backfill-space/datafile 2020-09-12T14:32:52.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:52.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-511 td/osd-backfill-space/datafile 2020-09-12T14:32:52.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:52.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-512 td/osd-backfill-space/datafile 2020-09-12T14:32:52.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:52.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-513 td/osd-backfill-space/datafile 2020-09-12T14:32:52.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:52.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-514 td/osd-backfill-space/datafile 2020-09-12T14:32:52.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:52.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-515 td/osd-backfill-space/datafile 2020-09-12T14:32:52.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:52.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-516 td/osd-backfill-space/datafile 2020-09-12T14:32:52.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:52.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-517 td/osd-backfill-space/datafile 2020-09-12T14:32:52.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:52.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-518 td/osd-backfill-space/datafile 2020-09-12T14:32:52.864 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:52.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-519 td/osd-backfill-space/datafile 2020-09-12T14:32:52.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:52.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-520 td/osd-backfill-space/datafile 2020-09-12T14:32:53.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:53.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-521 td/osd-backfill-space/datafile 2020-09-12T14:32:53.090 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:53.091 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-522 td/osd-backfill-space/datafile 2020-09-12T14:32:53.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:53.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-523 td/osd-backfill-space/datafile 2020-09-12T14:32:53.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:53.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-524 td/osd-backfill-space/datafile 2020-09-12T14:32:53.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:53.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-525 td/osd-backfill-space/datafile 2020-09-12T14:32:53.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:53.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-526 td/osd-backfill-space/datafile 2020-09-12T14:32:53.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:53.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-527 td/osd-backfill-space/datafile 2020-09-12T14:32:53.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:53.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-528 td/osd-backfill-space/datafile 2020-09-12T14:32:53.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:53.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-529 td/osd-backfill-space/datafile 2020-09-12T14:32:53.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:53.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-530 td/osd-backfill-space/datafile 2020-09-12T14:32:53.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:53.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-531 td/osd-backfill-space/datafile 2020-09-12T14:32:53.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:53.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-532 td/osd-backfill-space/datafile 2020-09-12T14:32:53.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:53.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-533 td/osd-backfill-space/datafile 2020-09-12T14:32:53.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:53.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-534 td/osd-backfill-space/datafile 2020-09-12T14:32:54.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:54.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-535 td/osd-backfill-space/datafile 2020-09-12T14:32:54.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:54.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-536 td/osd-backfill-space/datafile 2020-09-12T14:32:54.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:54.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-537 td/osd-backfill-space/datafile 2020-09-12T14:32:54.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:54.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-538 td/osd-backfill-space/datafile 2020-09-12T14:32:54.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:54.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-539 td/osd-backfill-space/datafile 2020-09-12T14:32:54.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:54.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-540 td/osd-backfill-space/datafile 2020-09-12T14:32:54.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:54.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-541 td/osd-backfill-space/datafile 2020-09-12T14:32:54.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:54.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-542 td/osd-backfill-space/datafile 2020-09-12T14:32:54.765 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:54.765 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-543 td/osd-backfill-space/datafile 2020-09-12T14:32:54.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:54.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-544 td/osd-backfill-space/datafile 2020-09-12T14:32:54.915 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:54.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-545 td/osd-backfill-space/datafile 2020-09-12T14:32:54.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:54.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-546 td/osd-backfill-space/datafile 2020-09-12T14:32:55.066 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:55.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-547 td/osd-backfill-space/datafile 2020-09-12T14:32:55.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:55.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-548 td/osd-backfill-space/datafile 2020-09-12T14:32:55.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:55.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-549 td/osd-backfill-space/datafile 2020-09-12T14:32:55.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:55.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-550 td/osd-backfill-space/datafile 2020-09-12T14:32:55.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:55.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-551 td/osd-backfill-space/datafile 2020-09-12T14:32:55.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:55.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-552 td/osd-backfill-space/datafile 2020-09-12T14:32:55.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:55.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-553 td/osd-backfill-space/datafile 2020-09-12T14:32:55.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:55.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-554 td/osd-backfill-space/datafile 2020-09-12T14:32:55.668 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:55.669 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-555 td/osd-backfill-space/datafile 2020-09-12T14:32:55.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:55.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-556 td/osd-backfill-space/datafile 2020-09-12T14:32:55.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:55.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-557 td/osd-backfill-space/datafile 2020-09-12T14:32:55.894 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:55.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-558 td/osd-backfill-space/datafile 2020-09-12T14:32:55.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:55.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-559 td/osd-backfill-space/datafile 2020-09-12T14:32:56.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:56.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-560 td/osd-backfill-space/datafile 2020-09-12T14:32:56.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:56.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-561 td/osd-backfill-space/datafile 2020-09-12T14:32:56.262 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:56.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-562 td/osd-backfill-space/datafile 2020-09-12T14:32:56.371 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:56.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-563 td/osd-backfill-space/datafile 2020-09-12T14:32:56.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:56.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-564 td/osd-backfill-space/datafile 2020-09-12T14:32:56.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:56.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-565 td/osd-backfill-space/datafile 2020-09-12T14:32:56.597 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:56.597 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-566 td/osd-backfill-space/datafile 2020-09-12T14:32:56.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:56.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-567 td/osd-backfill-space/datafile 2020-09-12T14:32:56.765 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:56.765 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-568 td/osd-backfill-space/datafile 2020-09-12T14:32:56.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:56.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-569 td/osd-backfill-space/datafile 2020-09-12T14:32:56.915 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:56.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-570 td/osd-backfill-space/datafile 2020-09-12T14:32:56.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:56.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-571 td/osd-backfill-space/datafile 2020-09-12T14:32:57.066 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:57.066 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-572 td/osd-backfill-space/datafile 2020-09-12T14:32:57.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:57.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-573 td/osd-backfill-space/datafile 2020-09-12T14:32:57.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:57.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-574 td/osd-backfill-space/datafile 2020-09-12T14:32:57.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:57.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-575 td/osd-backfill-space/datafile 2020-09-12T14:32:57.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:57.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-576 td/osd-backfill-space/datafile 2020-09-12T14:32:57.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:57.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-577 td/osd-backfill-space/datafile 2020-09-12T14:32:57.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:57.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-578 td/osd-backfill-space/datafile 2020-09-12T14:32:57.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:57.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-579 td/osd-backfill-space/datafile 2020-09-12T14:32:57.677 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:57.677 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-580 td/osd-backfill-space/datafile 2020-09-12T14:32:57.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:57.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-581 td/osd-backfill-space/datafile 2020-09-12T14:32:57.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:57.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-582 td/osd-backfill-space/datafile 2020-09-12T14:32:57.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:57.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-583 td/osd-backfill-space/datafile 2020-09-12T14:32:57.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:57.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-584 td/osd-backfill-space/datafile 2020-09-12T14:32:58.062 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:58.062 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-585 td/osd-backfill-space/datafile 2020-09-12T14:32:58.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:58.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-586 td/osd-backfill-space/datafile 2020-09-12T14:32:58.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:58.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-587 td/osd-backfill-space/datafile 2020-09-12T14:32:58.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:58.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-588 td/osd-backfill-space/datafile 2020-09-12T14:32:58.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:58.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-589 td/osd-backfill-space/datafile 2020-09-12T14:32:58.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:58.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-590 td/osd-backfill-space/datafile 2020-09-12T14:32:58.548 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:58.548 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-591 td/osd-backfill-space/datafile 2020-09-12T14:32:58.673 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:58.673 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-592 td/osd-backfill-space/datafile 2020-09-12T14:32:58.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:58.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-593 td/osd-backfill-space/datafile 2020-09-12T14:32:58.823 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:58.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-594 td/osd-backfill-space/datafile 2020-09-12T14:32:58.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:58.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-595 td/osd-backfill-space/datafile 2020-09-12T14:32:58.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:58.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-596 td/osd-backfill-space/datafile 2020-09-12T14:32:59.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:59.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-597 td/osd-backfill-space/datafile 2020-09-12T14:32:59.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:59.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-598 td/osd-backfill-space/datafile 2020-09-12T14:32:59.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:59.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-599 td/osd-backfill-space/datafile 2020-09-12T14:32:59.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:462: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:32:59.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:464: TEST_backfill_multi_partial: rados -p fillpool put obj-fill-600 td/osd-backfill-space/datafile 2020-09-12T14:32:59.385 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:467: TEST_backfill_multi_partial: get_primary fillpool obj-fill-1 2020-09-12T14:32:59.385 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1134: get_primary: local poolname=fillpool 2020-09-12T14:32:59.385 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1135: get_primary: local objectname=obj-fill-1 2020-09-12T14:32:59.385 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1137: get_primary: ceph --format json osd map fillpool obj-fill-1 2020-09-12T14:32:59.386 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1138: get_primary: jq .acting_primary 2020-09-12T14:32:59.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:467: TEST_backfill_multi_partial: local fillosd=1 2020-09-12T14:32:59.793 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:468: TEST_backfill_multi_partial: expr 1 + 1 2020-09-12T14:32:59.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:468: TEST_backfill_multi_partial: osd=2 2020-09-12T14:32:59.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:469: TEST_backfill_multi_partial: '[' 2 = 3 ']' 2020-09-12T14:32:59.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:473: TEST_backfill_multi_partial: kill_daemon td/osd-backfill-space/osd.1.pid TERM 2020-09-12T14:32:59.794 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:255: kill_daemon: cat td/osd-backfill-space/osd.1.pid 2020-09-12T14:32:59.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:255: kill_daemon: local pid=180332 2020-09-12T14:32:59.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:256: kill_daemon: local send_signal=TERM 2020-09-12T14:32:59.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:257: kill_daemon: local 'delays=0.1 0.2 1 1 1 2 3 5 5 5 10 10 20 60 60 60 120' 2020-09-12T14:32:59.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:258: kill_daemon: local exit_code=1 2020-09-12T14:32:59.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:261: kill_daemon: for try in $delays 0 2020-09-12T14:32:59.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: kill -TERM 180332 2020-09-12T14:32:59.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:263: kill_daemon: exit_code=1 2020-09-12T14:32:59.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:268: kill_daemon: send_signal=0 2020-09-12T14:32:59.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:269: kill_daemon: sleep 0.1 2020-09-12T14:32:59.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:261: kill_daemon: for try in $delays 0 2020-09-12T14:32:59.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: kill -0 180332 2020-09-12T14:32:59.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:265: kill_daemon: exit_code=0 2020-09-12T14:32:59.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:266: kill_daemon: break 2020-09-12T14:32:59.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:271: kill_daemon: return 0 2020-09-12T14:32:59.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:474: TEST_backfill_multi_partial: ceph osd out osd.1 2020-09-12T14:33:00.620 INFO:tasks.workunit.client.0.smithi027.stderr:osd.1 is already out. 2020-09-12T14:33:00.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:476: TEST_backfill_multi_partial: _objectstore_tool_nodown td/osd-backfill-space 1 --op export-remove --pgid 1.0 --file td/osd-backfill-space/fillexport.out 2020-09-12T14:33:00.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1195: _objectstore_tool_nodown: local dir=td/osd-backfill-space 2020-09-12T14:33:00.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: _objectstore_tool_nodown: shift 2020-09-12T14:33:00.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: _objectstore_tool_nodown: local id=1 2020-09-12T14:33:00.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1198: _objectstore_tool_nodown: shift 2020-09-12T14:33:00.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: _objectstore_tool_nodown: local osd_data=td/osd-backfill-space/1 2020-09-12T14:33:00.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1202: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-backfill-space/1 --op export-remove --pgid 1.0 --file td/osd-backfill-space/fillexport.out 2020-09-12T14:33:01.935 INFO:tasks.workunit.client.0.smithi027.stderr:Exporting 1.0 info 1.0( v 34'600 (34'500,34'600] local-lis/les=24/25 n=600 ec=21/21 lis/c=24/24 les/c/f=25/25/0 sis=24) 2020-09-12T14:33:02.145 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:0002d87b:::obj-fill-318:head# 2020-09-12T14:33:02.159 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:00676868:::obj-fill-583:head# 2020-09-12T14:33:02.188 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:00f4514e:::obj-fill-547:head# 2020-09-12T14:33:02.195 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:01596a3c:::obj-fill-533:head# 2020-09-12T14:33:02.220 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:0285e89c:::obj-fill-190:head# 2020-09-12T14:33:02.228 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:02bd591e:::obj-fill-463:head# 2020-09-12T14:33:02.236 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:02c01ffe:::obj-fill-28:head# 2020-09-12T14:33:02.236 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:03ada31e:::obj-fill-285:head# 2020-09-12T14:33:02.237 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:03b07c3f:::obj-fill-568:head# 2020-09-12T14:33:02.237 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:03b85770:::obj-fill-573:head# 2020-09-12T14:33:02.238 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:03dcbe89:::obj-fill-375:head# 2020-09-12T14:33:02.245 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:04231826:::obj-fill-594:head# 2020-09-12T14:33:02.247 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:04d48684:::obj-fill-421:head# 2020-09-12T14:33:02.247 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:05385fcc:::obj-fill-299:head# 2020-09-12T14:33:02.250 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:05785d38:::obj-fill-159:head# 2020-09-12T14:33:02.250 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:057ec231:::obj-fill-138:head# 2020-09-12T14:33:02.250 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:06058b5e:::obj-fill-569:head# 2020-09-12T14:33:02.252 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:06d2a3d9:::obj-fill-284:head# 2020-09-12T14:33:02.253 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:07597b63:::obj-fill-459:head# 2020-09-12T14:33:02.253 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:075f4069:::obj-fill-550:head# 2020-09-12T14:33:02.254 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:08234052:::obj-fill-17:head# 2020-09-12T14:33:02.254 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:091ce2ec:::obj-fill-472:head# 2020-09-12T14:33:02.255 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:095fd509:::obj-fill-210:head# 2020-09-12T14:33:02.255 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:0a52a594:::obj-fill-116:head# 2020-09-12T14:33:02.256 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:0a63716b:::obj-fill-225:head# 2020-09-12T14:33:02.256 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:0b0fa6fb:::obj-fill-541:head# 2020-09-12T14:33:02.257 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:0b673f72:::obj-fill-232:head# 2020-09-12T14:33:02.257 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:0bbcd1b2:::obj-fill-582:head# 2020-09-12T14:33:02.263 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:0caf8ce3:::obj-fill-214:head# 2020-09-12T14:33:02.264 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:0d94b9fb:::obj-fill-415:head# 2020-09-12T14:33:02.264 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:0de4e6f7:::obj-fill-94:head# 2020-09-12T14:33:02.271 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:0ef75768:::obj-fill-489:head# 2020-09-12T14:33:02.273 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:0f67284f:::obj-fill-213:head# 2020-09-12T14:33:02.274 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:0f823df2:::obj-fill-234:head# 2020-09-12T14:33:02.274 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:1005c08d:::obj-fill-490:head# 2020-09-12T14:33:02.275 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:1039525f:::obj-fill-369:head# 2020-09-12T14:33:02.275 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:105fb36c:::obj-fill-226:head# 2020-09-12T14:33:02.276 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:111abdeb:::obj-fill-554:head# 2020-09-12T14:33:02.276 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:11392f3d:::obj-fill-453:head# 2020-09-12T14:33:02.277 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:1169976c:::obj-fill-417:head# 2020-09-12T14:33:02.277 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:11840a2a:::obj-fill-477:head# 2020-09-12T14:33:02.278 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:11a4832e:::obj-fill-279:head# 2020-09-12T14:33:02.278 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:11a4d2f6:::obj-fill-256:head# 2020-09-12T14:33:02.279 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:12d52a31:::obj-fill-22:head# 2020-09-12T14:33:02.279 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:133c45db:::obj-fill-257:head# 2020-09-12T14:33:02.280 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:13563256:::obj-fill-150:head# 2020-09-12T14:33:02.280 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:13c09e00:::obj-fill-71:head# 2020-09-12T14:33:02.286 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:13e9fbed:::obj-fill-242:head# 2020-09-12T14:33:02.286 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:13fae6c2:::obj-fill-163:head# 2020-09-12T14:33:02.287 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:145025e0:::obj-fill-543:head# 2020-09-12T14:33:02.287 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:14517c84:::obj-fill-13:head# 2020-09-12T14:33:02.289 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:14802e9c:::obj-fill-127:head# 2020-09-12T14:33:02.290 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:150f1c96:::obj-fill-62:head# 2020-09-12T14:33:02.290 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:155e90a2:::obj-fill-537:head# 2020-09-12T14:33:02.291 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:1632c776:::obj-fill-262:head# 2020-09-12T14:33:02.291 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:163bde28:::obj-fill-523:head# 2020-09-12T14:33:02.292 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:169c4ad0:::obj-fill-85:head# 2020-09-12T14:33:02.292 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:16c4da1d:::obj-fill-436:head# 2020-09-12T14:33:02.293 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:17b28c32:::obj-fill-345:head# 2020-09-12T14:33:02.293 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:17b34255:::obj-fill-577:head# 2020-09-12T14:33:02.294 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:17d2541f:::obj-fill-426:head# 2020-09-12T14:33:02.294 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:17ea19f2:::obj-fill-542:head# 2020-09-12T14:33:02.294 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:189155ea:::obj-fill-496:head# 2020-09-12T14:33:02.295 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:191a6d81:::obj-fill-548:head# 2020-09-12T14:33:02.295 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:1a964993:::obj-fill-576:head# 2020-09-12T14:33:02.296 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:1aa623dc:::obj-fill-356:head# 2020-09-12T14:33:02.296 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:1abe5c38:::obj-fill-561:head# 2020-09-12T14:33:02.296 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:1bddc06b:::obj-fill-7:head# 2020-09-12T14:33:02.297 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:1be86b71:::obj-fill-295:head# 2020-09-12T14:33:02.297 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:1c499bca:::obj-fill-240:head# 2020-09-12T14:33:02.297 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:1c545e27:::obj-fill-509:head# 2020-09-12T14:33:02.298 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:1c63c8f4:::obj-fill-4:head# 2020-09-12T14:33:02.298 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:1d5a8f59:::obj-fill-377:head# 2020-09-12T14:33:02.299 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:1d85b5df:::obj-fill-201:head# 2020-09-12T14:33:02.320 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:1d9a0d5b:::obj-fill-581:head# 2020-09-12T14:33:02.328 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:1dcf5667:::obj-fill-488:head# 2020-09-12T14:33:02.329 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:1fa3090d:::obj-fill-458:head# 2020-09-12T14:33:02.345 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:1fe937f6:::obj-fill-337:head# 2020-09-12T14:33:02.353 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:1ff54542:::obj-fill-349:head# 2020-09-12T14:33:02.354 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:20d9e0ca:::obj-fill-367:head# 2020-09-12T14:33:02.354 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:21b8e3bb:::obj-fill-538:head# 2020-09-12T14:33:02.354 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:227183d4:::obj-fill-235:head# 2020-09-12T14:33:02.355 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:229e5e5d:::obj-fill-522:head# 2020-09-12T14:33:02.355 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:234f8861:::obj-fill-476:head# 2020-09-12T14:33:02.356 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:23ac2c9d:::obj-fill-471:head# 2020-09-12T14:33:02.356 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:25db1742:::obj-fill-200:head# 2020-09-12T14:33:02.356 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:264f8545:::obj-fill-230:head# 2020-09-12T14:33:02.357 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:267ba97f:::obj-fill-493:head# 2020-09-12T14:33:02.357 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:26b067e8:::obj-fill-208:head# 2020-09-12T14:33:02.358 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:26f521cb:::obj-fill-300:head# 2020-09-12T14:33:02.358 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:27e70a11:::obj-fill-368:head# 2020-09-12T14:33:02.359 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:29357d4a:::obj-fill-456:head# 2020-09-12T14:33:02.359 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:29f2567e:::obj-fill-414:head# 2020-09-12T14:33:02.359 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:2a10cbb6:::obj-fill-228:head# 2020-09-12T14:33:02.359 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:2a1519a6:::obj-fill-148:head# 2020-09-12T14:33:02.360 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:2a686ca6:::obj-fill-411:head# 2020-09-12T14:33:02.360 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:2ab3f384:::obj-fill-77:head# 2020-09-12T14:33:02.360 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:2ae6a1d0:::obj-fill-187:head# 2020-09-12T14:33:02.361 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:2b890959:::obj-fill-222:head# 2020-09-12T14:33:02.361 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:2bcc26c4:::obj-fill-114:head# 2020-09-12T14:33:02.361 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:2bd3a3a2:::obj-fill-281:head# 2020-09-12T14:33:02.362 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:2c1a21d7:::obj-fill-505:head# 2020-09-12T14:33:02.362 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:2c255347:::obj-fill-95:head# 2020-09-12T14:33:02.363 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:2c2a3f62:::obj-fill-131:head# 2020-09-12T14:33:02.363 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:2c7d92ce:::obj-fill-442:head# 2020-09-12T14:33:02.363 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:2d1ad117:::obj-fill-443:head# 2020-09-12T14:33:02.364 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:2d57b2f7:::obj-fill-192:head# 2020-09-12T14:33:02.364 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:2d7b53ea:::obj-fill-79:head# 2020-09-12T14:33:02.365 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:2db25266:::obj-fill-121:head# 2020-09-12T14:33:02.365 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:2e27853e:::obj-fill-473:head# 2020-09-12T14:33:02.366 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:2f12cb21:::obj-fill-386:head# 2020-09-12T14:33:02.366 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:2f957332:::obj-fill-422:head# 2020-09-12T14:33:02.367 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:2feae493:::obj-fill-301:head# 2020-09-12T14:33:02.367 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:302c95d2:::obj-fill-110:head# 2020-09-12T14:33:02.367 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:302d6b76:::obj-fill-374:head# 2020-09-12T14:33:02.368 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:30581f34:::obj-fill-313:head# 2020-09-12T14:33:02.368 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:307e86ac:::obj-fill-381:head# 2020-09-12T14:33:02.368 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:30ab77bf:::obj-fill-275:head# 2020-09-12T14:33:02.369 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:31127e6d:::obj-fill-250:head# 2020-09-12T14:33:02.369 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:32c93500:::obj-fill-155:head# 2020-09-12T14:33:02.370 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:339880b7:::obj-fill-227:head# 2020-09-12T14:33:02.370 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:34069e1e:::obj-fill-292:head# 2020-09-12T14:33:02.370 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:346dff31:::obj-fill-157:head# 2020-09-12T14:33:02.371 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:34ade0f1:::obj-fill-575:head# 2020-09-12T14:33:02.371 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:34bbae43:::obj-fill-90:head# 2020-09-12T14:33:02.371 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:34e97cf8:::obj-fill-334:head# 2020-09-12T14:33:02.372 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:35d7ccc2:::obj-fill-510:head# 2020-09-12T14:33:02.372 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:35f519b6:::obj-fill-513:head# 2020-09-12T14:33:02.372 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:364727e8:::obj-fill-130:head# 2020-09-12T14:33:02.373 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:364c0361:::obj-fill-586:head# 2020-09-12T14:33:02.373 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:368e7baf:::obj-fill-546:head# 2020-09-12T14:33:02.373 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:36ffb224:::obj-fill-82:head# 2020-09-12T14:33:02.374 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:371cb800:::obj-fill-330:head# 2020-09-12T14:33:02.374 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:37d5378d:::obj-fill-175:head# 2020-09-12T14:33:02.375 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:385593f3:::obj-fill-325:head# 2020-09-12T14:33:02.375 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:388d5073:::obj-fill-317:head# 2020-09-12T14:33:02.376 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:394af079:::obj-fill-59:head# 2020-09-12T14:33:02.376 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:3a41ad37:::obj-fill-409:head# 2020-09-12T14:33:02.376 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:3b3c63d2:::obj-fill-136:head# 2020-09-12T14:33:02.377 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:3bb50131:::obj-fill-512:head# 2020-09-12T14:33:02.377 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:3bb52bb5:::obj-fill-188:head# 2020-09-12T14:33:02.378 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:3c0683d1:::obj-fill-372:head# 2020-09-12T14:33:02.378 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:3c67d3f4:::obj-fill-249:head# 2020-09-12T14:33:02.379 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:3c6a4a4c:::obj-fill-495:head# 2020-09-12T14:33:02.379 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:3c7fc8c9:::obj-fill-324:head# 2020-09-12T14:33:02.380 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:3d940c50:::obj-fill-591:head# 2020-09-12T14:33:02.380 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:3ed5508e:::obj-fill-558:head# 2020-09-12T14:33:02.381 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:3f598316:::obj-fill-389:head# 2020-09-12T14:33:02.381 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:401cd0ce:::obj-fill-251:head# 2020-09-12T14:33:02.381 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:40fae006:::obj-fill-14:head# 2020-09-12T14:33:02.381 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:4120b10f:::obj-fill-165:head# 2020-09-12T14:33:02.382 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:41b382c2:::obj-fill-289:head# 2020-09-12T14:33:02.382 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:4277cdea:::obj-fill-319:head# 2020-09-12T14:33:02.383 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:42978c3c:::obj-fill-174:head# 2020-09-12T14:33:02.383 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:42b08dc2:::obj-fill-553:head# 2020-09-12T14:33:02.383 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:42ec4bf6:::obj-fill-209:head# 2020-09-12T14:33:02.384 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:430c129d:::obj-fill-448:head# 2020-09-12T14:33:02.384 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:4335d553:::obj-fill-600:head# 2020-09-12T14:33:02.384 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:44804f62:::obj-fill-532:head# 2020-09-12T14:33:02.385 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:44954d54:::obj-fill-2:head# 2020-09-12T14:33:02.385 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:4502f904:::obj-fill-429:head# 2020-09-12T14:33:02.386 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:4568ddcb:::obj-fill-382:head# 2020-09-12T14:33:02.386 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:45939974:::obj-fill-563:head# 2020-09-12T14:33:02.386 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:462a5bbd:::obj-fill-567:head# 2020-09-12T14:33:02.387 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:47773ae4:::obj-fill-307:head# 2020-09-12T14:33:02.387 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:4802b263:::obj-fill-387:head# 2020-09-12T14:33:02.388 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:483d938e:::obj-fill-128:head# 2020-09-12T14:33:02.388 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:48a7ad29:::obj-fill-147:head# 2020-09-12T14:33:02.388 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:493b61ce:::obj-fill-454:head# 2020-09-12T14:33:02.388 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:4afb70fe:::obj-fill-186:head# 2020-09-12T14:33:02.389 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:4bff08e3:::obj-fill-352:head# 2020-09-12T14:33:02.390 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:4c01eabe:::obj-fill-370:head# 2020-09-12T14:33:02.390 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:4c131560:::obj-fill-6:head# 2020-09-12T14:33:02.390 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:4c4c5e14:::obj-fill-350:head# 2020-09-12T14:33:02.391 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:4c80dc2c:::obj-fill-357:head# 2020-09-12T14:33:02.391 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:4c9dee9b:::obj-fill-540:head# 2020-09-12T14:33:02.392 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:4ce40c18:::obj-fill-280:head# 2020-09-12T14:33:02.392 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:4d854fc3:::obj-fill-585:head# 2020-09-12T14:33:02.392 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:4f615b53:::obj-fill-101:head# 2020-09-12T14:33:02.393 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5042b30f:::obj-fill-84:head# 2020-09-12T14:33:02.393 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:507dc08b:::obj-fill-336:head# 2020-09-12T14:33:02.394 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:512fcf5a:::obj-fill-169:head# 2020-09-12T14:33:02.394 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:51a0c8f8:::obj-fill-578:head# 2020-09-12T14:33:02.395 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:51fd05a2:::obj-fill-425:head# 2020-09-12T14:33:02.395 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:52400ebc:::obj-fill-81:head# 2020-09-12T14:33:02.395 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5288d40c:::obj-fill-16:head# 2020-09-12T14:33:02.396 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5295b0ee:::obj-fill-224:head# 2020-09-12T14:33:02.396 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:531e4c60:::obj-fill-103:head# 2020-09-12T14:33:02.396 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:54259f28:::obj-fill-584:head# 2020-09-12T14:33:02.397 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:542d619c:::obj-fill-528:head# 2020-09-12T14:33:02.398 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5503ca3e:::obj-fill-494:head# 2020-09-12T14:33:02.398 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5555c48b:::obj-fill-403:head# 2020-09-12T14:33:02.399 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:558d1ec7:::obj-fill-447:head# 2020-09-12T14:33:02.399 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:55aa2233:::obj-fill-21:head# 2020-09-12T14:33:02.399 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:55aaf8c4:::obj-fill-29:head# 2020-09-12T14:33:02.400 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:55e246e1:::obj-fill-65:head# 2020-09-12T14:33:02.400 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:563b14a8:::obj-fill-120:head# 2020-09-12T14:33:02.400 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:56e1df25:::obj-fill-305:head# 2020-09-12T14:33:02.401 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5702979c:::obj-fill-73:head# 2020-09-12T14:33:02.401 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:570722ba:::obj-fill-219:head# 2020-09-12T14:33:02.401 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:57f7a09b:::obj-fill-571:head# 2020-09-12T14:33:02.402 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:585309ba:::obj-fill-144:head# 2020-09-12T14:33:02.402 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:585c7c9c:::obj-fill-437:head# 2020-09-12T14:33:02.403 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:587c63db:::obj-fill-182:head# 2020-09-12T14:33:02.403 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:58f34d6b:::obj-fill-598:head# 2020-09-12T14:33:02.403 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:58f40088:::obj-fill-298:head# 2020-09-12T14:33:02.404 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:591e150c:::obj-fill-32:head# 2020-09-12T14:33:02.404 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:59971ddb:::obj-fill-559:head# 2020-09-12T14:33:02.404 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5adafca4:::obj-fill-56:head# 2020-09-12T14:33:02.404 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5b32b04f:::obj-fill-484:head# 2020-09-12T14:33:02.405 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5bdaeb16:::obj-fill-99:head# 2020-09-12T14:33:02.405 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5c1d6f4b:::obj-fill-133:head# 2020-09-12T14:33:02.405 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5c92b8f1:::obj-fill-255:head# 2020-09-12T14:33:02.406 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5ca4850f:::obj-fill-238:head# 2020-09-12T14:33:02.406 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5d928a5b:::obj-fill-395:head# 2020-09-12T14:33:02.406 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5dd49320:::obj-fill-423:head# 2020-09-12T14:33:02.407 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5e7bc368:::obj-fill-506:head# 2020-09-12T14:33:02.407 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5ebd5ee3:::obj-fill-322:head# 2020-09-12T14:33:02.408 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5edeedf1:::obj-fill-557:head# 2020-09-12T14:33:02.408 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:5facf8e2:::obj-fill-15:head# 2020-09-12T14:33:02.409 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:603a2c76:::obj-fill-439:head# 2020-09-12T14:33:02.409 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:60b30714:::obj-fill-53:head# 2020-09-12T14:33:02.409 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:612b4a8a:::obj-fill-412:head# 2020-09-12T14:33:02.410 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:61a111b6:::obj-fill-87:head# 2020-09-12T14:33:02.410 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:63a0e8a2:::obj-fill-266:head# 2020-09-12T14:33:02.410 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:63aa4d60:::obj-fill-68:head# 2020-09-12T14:33:02.410 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:63ac5b95:::obj-fill-405:head# 2020-09-12T14:33:02.411 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:6415a9e3:::obj-fill-400:head# 2020-09-12T14:33:02.411 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:64ba03ae:::obj-fill-20:head# 2020-09-12T14:33:02.412 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:6512f015:::obj-fill-597:head# 2020-09-12T14:33:02.412 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:65433104:::obj-fill-504:head# 2020-09-12T14:33:02.413 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:65950c58:::obj-fill-86:head# 2020-09-12T14:33:02.413 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:65b6b2d7:::obj-fill-39:head# 2020-09-12T14:33:02.413 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:661c8afd:::obj-fill-267:head# 2020-09-12T14:33:02.414 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:66f87c94:::obj-fill-229:head# 2020-09-12T14:33:02.414 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:67292cef:::obj-fill-93:head# 2020-09-12T14:33:02.415 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:67f0e12c:::obj-fill-363:head# 2020-09-12T14:33:02.415 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:696eb83b:::obj-fill-185:head# 2020-09-12T14:33:02.415 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:6b33981c:::obj-fill-259:head# 2020-09-12T14:33:02.416 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:6b5a527c:::obj-fill-485:head# 2020-09-12T14:33:02.416 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:6b6f1852:::obj-fill-390:head# 2020-09-12T14:33:02.417 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:6ba5c5f8:::obj-fill-304:head# 2020-09-12T14:33:02.417 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:6bce929d:::obj-fill-518:head# 2020-09-12T14:33:02.417 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:6c8ffc04:::obj-fill-61:head# 2020-09-12T14:33:02.418 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:6ca7e551:::obj-fill-40:head# 2020-09-12T14:33:02.418 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:6caff2cd:::obj-fill-332:head# 2020-09-12T14:33:02.419 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:6ce4adc7:::obj-fill-574:head# 2020-09-12T14:33:02.419 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:6d09d770:::obj-fill-74:head# 2020-09-12T14:33:02.419 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:6e039822:::obj-fill-278:head# 2020-09-12T14:33:02.420 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:6f2cd109:::obj-fill-297:head# 2020-09-12T14:33:02.420 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:6f3754c4:::obj-fill-88:head# 2020-09-12T14:33:02.420 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:6f86cfd2:::obj-fill-511:head# 2020-09-12T14:33:02.420 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:6fe3f3f7:::obj-fill-193:head# 2020-09-12T14:33:02.421 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7002c57d:::obj-fill-50:head# 2020-09-12T14:33:02.421 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7030b2a3:::obj-fill-217:head# 2020-09-12T14:33:02.421 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:70665087:::obj-fill-51:head# 2020-09-12T14:33:02.422 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:70a0f5e0:::obj-fill-11:head# 2020-09-12T14:33:02.422 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:71f4643d:::obj-fill-236:head# 2020-09-12T14:33:02.423 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:722fbd0f:::obj-fill-302:head# 2020-09-12T14:33:02.423 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:72584da9:::obj-fill-445:head# 2020-09-12T14:33:02.423 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:72d91d24:::obj-fill-218:head# 2020-09-12T14:33:02.424 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:72f7268b:::obj-fill-106:head# 2020-09-12T14:33:02.424 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:74077abf:::obj-fill-435:head# 2020-09-12T14:33:02.424 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:740ca050:::obj-fill-112:head# 2020-09-12T14:33:02.424 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7435e881:::obj-fill-78:head# 2020-09-12T14:33:02.425 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:74ba0bb7:::obj-fill-478:head# 2020-09-12T14:33:02.425 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:74d82091:::obj-fill-96:head# 2020-09-12T14:33:02.425 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7515f56c:::obj-fill-483:head# 2020-09-12T14:33:02.426 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7550d6fe:::obj-fill-286:head# 2020-09-12T14:33:02.426 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:75807edc:::obj-fill-223:head# 2020-09-12T14:33:02.426 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:762ba324:::obj-fill-172:head# 2020-09-12T14:33:02.427 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:76369bda:::obj-fill-570:head# 2020-09-12T14:33:02.427 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7667a5b0:::obj-fill-196:head# 2020-09-12T14:33:02.427 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:77736c0c:::obj-fill-566:head# 2020-09-12T14:33:02.428 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:77fb47a4:::obj-fill-314:head# 2020-09-12T14:33:02.428 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:785ad894:::obj-fill-80:head# 2020-09-12T14:33:02.428 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:785e3d14:::obj-fill-530:head# 2020-09-12T14:33:02.428 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:788efef4:::obj-fill-595:head# 2020-09-12T14:33:02.429 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:789ec5ab:::obj-fill-44:head# 2020-09-12T14:33:02.429 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:798a3fa8:::obj-fill-517:head# 2020-09-12T14:33:02.430 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:79a105ef:::obj-fill-315:head# 2020-09-12T14:33:02.430 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:79d02f04:::obj-fill-243:head# 2020-09-12T14:33:02.430 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7a00dfcb:::obj-fill-402:head# 2020-09-12T14:33:02.431 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7b982684:::obj-fill-168:head# 2020-09-12T14:33:02.431 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7bb36851:::obj-fill-440:head# 2020-09-12T14:33:02.432 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7bb49d8c:::obj-fill-55:head# 2020-09-12T14:33:02.432 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7c77b783:::obj-fill-580:head# 2020-09-12T14:33:02.432 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7c89d75b:::obj-fill-555:head# 2020-09-12T14:33:02.433 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7d12fb67:::obj-fill-399:head# 2020-09-12T14:33:02.433 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7d139961:::obj-fill-92:head# 2020-09-12T14:33:02.434 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7d39ffd0:::obj-fill-135:head# 2020-09-12T14:33:02.434 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7e534cc4:::obj-fill-282:head# 2020-09-12T14:33:02.435 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7e7a08e0:::obj-fill-108:head# 2020-09-12T14:33:02.435 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7e8e2b67:::obj-fill-244:head# 2020-09-12T14:33:02.435 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7eea215e:::obj-fill-139:head# 2020-09-12T14:33:02.436 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7efa3c6e:::obj-fill-408:head# 2020-09-12T14:33:02.436 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7f03a24b:::obj-fill-45:head# 2020-09-12T14:33:02.436 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7f6a2e2b:::obj-fill-427:head# 2020-09-12T14:33:02.437 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7fb0abd6:::obj-fill-479:head# 2020-09-12T14:33:02.443 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:7fb77de1:::obj-fill-167:head# 2020-09-12T14:33:02.444 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:80012a03:::obj-fill-105:head# 2020-09-12T14:33:02.444 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:80075ae5:::obj-fill-376:head# 2020-09-12T14:33:02.445 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8089baa5:::obj-fill-97:head# 2020-09-12T14:33:02.445 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:80d0b4c5:::obj-fill-123:head# 2020-09-12T14:33:02.445 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:810f319f:::obj-fill-552:head# 2020-09-12T14:33:02.446 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:81427ef2:::obj-fill-474:head# 2020-09-12T14:33:02.446 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:818d780c:::obj-fill-33:head# 2020-09-12T14:33:02.447 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:819f31b9:::obj-fill-366:head# 2020-09-12T14:33:02.447 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8205549d:::obj-fill-125:head# 2020-09-12T14:33:02.448 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:821f56e5:::obj-fill-339:head# 2020-09-12T14:33:02.448 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:822807c6:::obj-fill-69:head# 2020-09-12T14:33:02.449 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:822fcf61:::obj-fill-560:head# 2020-09-12T14:33:02.449 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:82ad696b:::obj-fill-149:head# 2020-09-12T14:33:02.449 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:835cc771:::obj-fill-446:head# 2020-09-12T14:33:02.450 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8386bee1:::obj-fill-137:head# 2020-09-12T14:33:02.450 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:83c1303a:::obj-fill-311:head# 2020-09-12T14:33:02.450 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:83cf1c2a:::obj-fill-360:head# 2020-09-12T14:33:02.451 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:83d70321:::obj-fill-351:head# 2020-09-12T14:33:02.451 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:83ef162d:::obj-fill-331:head# 2020-09-12T14:33:02.451 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:840f8f20:::obj-fill-398:head# 2020-09-12T14:33:02.452 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:84d2e456:::obj-fill-207:head# 2020-09-12T14:33:02.452 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:856697ff:::obj-fill-333:head# 2020-09-12T14:33:02.453 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:858be937:::obj-fill-418:head# 2020-09-12T14:33:02.453 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:85a0b302:::obj-fill-438:head# 2020-09-12T14:33:02.453 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:85fa942e:::obj-fill-272:head# 2020-09-12T14:33:02.454 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:863eb6f8:::obj-fill-320:head# 2020-09-12T14:33:02.454 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:868d4588:::obj-fill-19:head# 2020-09-12T14:33:02.454 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8759d023:::obj-fill-466:head# 2020-09-12T14:33:02.454 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:88072111:::obj-fill-312:head# 2020-09-12T14:33:02.455 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:886edcef:::obj-fill-380:head# 2020-09-12T14:33:02.455 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:88ce753e:::obj-fill-43:head# 2020-09-12T14:33:02.455 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:890e1d83:::obj-fill-590:head# 2020-09-12T14:33:02.456 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:891566a3:::obj-fill-341:head# 2020-09-12T14:33:02.456 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:891ca235:::obj-fill-177:head# 2020-09-12T14:33:02.457 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:897fa9d0:::obj-fill-283:head# 2020-09-12T14:33:02.457 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:89cd304b:::obj-fill-592:head# 2020-09-12T14:33:02.457 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:89d04c4a:::obj-fill-388:head# 2020-09-12T14:33:02.458 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:89db45d9:::obj-fill-247:head# 2020-09-12T14:33:02.458 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8ad879c5:::obj-fill-492:head# 2020-09-12T14:33:02.458 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8af4b5de:::obj-fill-37:head# 2020-09-12T14:33:02.459 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8b132e86:::obj-fill-35:head# 2020-09-12T14:33:02.459 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8b80a60a:::obj-fill-100:head# 2020-09-12T14:33:02.459 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8bb64fa6:::obj-fill-599:head# 2020-09-12T14:33:02.460 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8bed03f5:::obj-fill-5:head# 2020-09-12T14:33:02.460 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8c60e135:::obj-fill-470:head# 2020-09-12T14:33:02.460 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8c774623:::obj-fill-38:head# 2020-09-12T14:33:02.461 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8cc528ae:::obj-fill-519:head# 2020-09-12T14:33:02.461 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8cf73826:::obj-fill-460:head# 2020-09-12T14:33:02.461 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8d3564b3:::obj-fill-452:head# 2020-09-12T14:33:02.461 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8dfe1ca2:::obj-fill-469:head# 2020-09-12T14:33:02.462 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8e527672:::obj-fill-246:head# 2020-09-12T14:33:02.462 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8e96103f:::obj-fill-444:head# 2020-09-12T14:33:02.463 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8ed6417f:::obj-fill-146:head# 2020-09-12T14:33:02.463 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8eda2ffc:::obj-fill-539:head# 2020-09-12T14:33:02.463 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8f08dd48:::obj-fill-26:head# 2020-09-12T14:33:02.464 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8f111b25:::obj-fill-18:head# 2020-09-12T14:33:02.464 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:8fde63f8:::obj-fill-338:head# 2020-09-12T14:33:02.465 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:9030746d:::obj-fill-468:head# 2020-09-12T14:33:02.465 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:9115d8a1:::obj-fill-260:head# 2020-09-12T14:33:02.465 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:91649909:::obj-fill-140:head# 2020-09-12T14:33:02.466 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:91c1081d:::obj-fill-404:head# 2020-09-12T14:33:02.466 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:91f42503:::obj-fill-216:head# 2020-09-12T14:33:02.467 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:927b34a0:::obj-fill-63:head# 2020-09-12T14:33:02.467 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:92a00a91:::obj-fill-467:head# 2020-09-12T14:33:02.467 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:930a7268:::obj-fill-450:head# 2020-09-12T14:33:02.468 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:93c3d0bf:::obj-fill-118:head# 2020-09-12T14:33:02.468 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:93ca7c9e:::obj-fill-132:head# 2020-09-12T14:33:02.468 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:942435d5:::obj-fill-378:head# 2020-09-12T14:33:02.469 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:9445f8c5:::obj-fill-245:head# 2020-09-12T14:33:02.469 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:9457c8d9:::obj-fill-535:head# 2020-09-12T14:33:02.470 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:950dc56e:::obj-fill-348:head# 2020-09-12T14:33:02.470 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:9559ee1d:::obj-fill-545:head# 2020-09-12T14:33:02.470 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:957c5cc4:::obj-fill-212:head# 2020-09-12T14:33:02.471 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:95bdac02:::obj-fill-588:head# 2020-09-12T14:33:02.471 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:95f202af:::obj-fill-126:head# 2020-09-12T14:33:02.472 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:96789003:::obj-fill-83:head# 2020-09-12T14:33:02.472 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:96d494a8:::obj-fill-316:head# 2020-09-12T14:33:02.472 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:97adb85b:::obj-fill-233:head# 2020-09-12T14:33:02.473 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:980d998c:::obj-fill-197:head# 2020-09-12T14:33:02.473 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:98235c44:::obj-fill-396:head# 2020-09-12T14:33:02.473 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:987d3b87:::obj-fill-327:head# 2020-09-12T14:33:02.474 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:990a5e24:::obj-fill-515:head# 2020-09-12T14:33:02.474 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:995fec05:::obj-fill-191:head# 2020-09-12T14:33:02.475 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:9a769b75:::obj-fill-587:head# 2020-09-12T14:33:02.475 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:9b141990:::obj-fill-394:head# 2020-09-12T14:33:02.476 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:9bb108fb:::obj-fill-579:head# 2020-09-12T14:33:02.476 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:9ce19d92:::obj-fill-361:head# 2020-09-12T14:33:02.476 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:9d8c3f95:::obj-fill-572:head# 2020-09-12T14:33:02.477 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:9e283e00:::obj-fill-254:head# 2020-09-12T14:33:02.477 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:9e3a1a61:::obj-fill-58:head# 2020-09-12T14:33:02.478 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:9e8f6029:::obj-fill-41:head# 2020-09-12T14:33:02.478 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:9ef117aa:::obj-fill-520:head# 2020-09-12T14:33:02.478 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:9f3bdfc2:::obj-fill-184:head# 2020-09-12T14:33:02.479 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:9fd89320:::obj-fill-364:head# 2020-09-12T14:33:02.479 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:9feb7062:::obj-fill-419:head# 2020-09-12T14:33:02.479 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a07504c4:::obj-fill-516:head# 2020-09-12T14:33:02.480 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a1897e34:::obj-fill-153:head# 2020-09-12T14:33:02.480 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a19a9d76:::obj-fill-30:head# 2020-09-12T14:33:02.481 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a1a49c98:::obj-fill-98:head# 2020-09-12T14:33:02.481 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a1ea911a:::obj-fill-36:head# 2020-09-12T14:33:02.481 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a24bad98:::obj-fill-335:head# 2020-09-12T14:33:02.482 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a263b16f:::obj-fill-326:head# 2020-09-12T14:33:02.482 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a27d661e:::obj-fill-152:head# 2020-09-12T14:33:02.482 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a282a129:::obj-fill-451:head# 2020-09-12T14:33:02.483 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a28f05ba:::obj-fill-241:head# 2020-09-12T14:33:02.483 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a2cdc5cc:::obj-fill-180:head# 2020-09-12T14:33:02.484 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a30777ea:::obj-fill-486:head# 2020-09-12T14:33:02.484 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a3a65661:::obj-fill-151:head# 2020-09-12T14:33:02.484 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a3cb2967:::obj-fill-195:head# 2020-09-12T14:33:02.485 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a3ffc098:::obj-fill-293:head# 2020-09-12T14:33:02.485 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a4a31ccc:::obj-fill-413:head# 2020-09-12T14:33:02.486 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a51c3677:::obj-fill-384:head# 2020-09-12T14:33:02.486 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a5e941a7:::obj-fill-141:head# 2020-09-12T14:33:02.486 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a6cdd8b2:::obj-fill-342:head# 2020-09-12T14:33:02.487 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a6e20b86:::obj-fill-593:head# 2020-09-12T14:33:02.487 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a7a9ccf9:::obj-fill-129:head# 2020-09-12T14:33:02.488 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a7b702d3:::obj-fill-49:head# 2020-09-12T14:33:02.488 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a8daf8bd:::obj-fill-391:head# 2020-09-12T14:33:02.488 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:a99c3cef:::obj-fill-171:head# 2020-09-12T14:33:02.488 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:aabb5718:::obj-fill-521:head# 2020-09-12T14:33:02.489 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ab0cb851:::obj-fill-183:head# 2020-09-12T14:33:02.489 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ab7a05e4:::obj-fill-203:head# 2020-09-12T14:33:02.490 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ab7dda83:::obj-fill-113:head# 2020-09-12T14:33:02.490 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:abb3502c:::obj-fill-461:head# 2020-09-12T14:33:02.490 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ac1c41be:::obj-fill-162:head# 2020-09-12T14:33:02.491 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ac2114ba:::obj-fill-239:head# 2020-09-12T14:33:02.491 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ad355fca:::obj-fill-271:head# 2020-09-12T14:33:02.491 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ad4634a0:::obj-fill-487:head# 2020-09-12T14:33:02.492 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:add2bb13:::obj-fill-31:head# 2020-09-12T14:33:02.492 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:aeb6258d:::obj-fill-124:head# 2020-09-12T14:33:02.493 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:af17f7d7:::obj-fill-464:head# 2020-09-12T14:33:02.493 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:af1e1cf0:::obj-fill-237:head# 2020-09-12T14:33:02.493 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:afc439ce:::obj-fill-248:head# 2020-09-12T14:33:02.493 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:afceab4d:::obj-fill-527:head# 2020-09-12T14:33:02.494 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b03af40c:::obj-fill-143:head# 2020-09-12T14:33:02.494 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b04cf0db:::obj-fill-507:head# 2020-09-12T14:33:02.495 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b07896f0:::obj-fill-269:head# 2020-09-12T14:33:02.495 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b0b954d5:::obj-fill-524:head# 2020-09-12T14:33:02.495 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b1cbfcb7:::obj-fill-134:head# 2020-09-12T14:33:02.496 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b2978fd3:::obj-fill-27:head# 2020-09-12T14:33:02.496 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b394798f:::obj-fill-156:head# 2020-09-12T14:33:02.497 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b3c6f36d:::obj-fill-401:head# 2020-09-12T14:33:02.497 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b3f8e5ac:::obj-fill-290:head# 2020-09-12T14:33:02.497 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b41ed5f8:::obj-fill-76:head# 2020-09-12T14:33:02.497 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b4878e76:::obj-fill-253:head# 2020-09-12T14:33:02.498 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b4e19741:::obj-fill-503:head# 2020-09-12T14:33:02.498 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b53a3690:::obj-fill-264:head# 2020-09-12T14:33:02.499 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b545ab4f:::obj-fill-24:head# 2020-09-12T14:33:02.499 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b55af5e6:::obj-fill-491:head# 2020-09-12T14:33:02.499 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b57edf82:::obj-fill-526:head# 2020-09-12T14:33:02.500 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b589d2be:::obj-fill-323:head# 2020-09-12T14:33:02.500 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b6207f32:::obj-fill-344:head# 2020-09-12T14:33:02.501 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b757ee2e:::obj-fill-433:head# 2020-09-12T14:33:02.501 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b75cc479:::obj-fill-34:head# 2020-09-12T14:33:02.501 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b7b03117:::obj-fill-42:head# 2020-09-12T14:33:02.501 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b7e6205d:::obj-fill-501:head# 2020-09-12T14:33:02.502 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b8d18b7a:::obj-fill-353:head# 2020-09-12T14:33:02.502 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b9189c29:::obj-fill-328:head# 2020-09-12T14:33:02.502 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b98e07aa:::obj-fill-54:head# 2020-09-12T14:33:02.503 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:b9cd1e9a:::obj-fill-371:head# 2020-09-12T14:33:02.503 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:bc3322bb:::obj-fill-340:head# 2020-09-12T14:33:02.503 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:bcf43161:::obj-fill-294:head# 2020-09-12T14:33:02.504 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:bd07fc4f:::obj-fill-549:head# 2020-09-12T14:33:02.504 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:bdf48e47:::obj-fill-430:head# 2020-09-12T14:33:02.505 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:be01c111:::obj-fill-1:head# 2020-09-12T14:33:02.505 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:be3a4bc7:::obj-fill-102:head# 2020-09-12T14:33:02.505 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:be8e1fcd:::obj-fill-321:head# 2020-09-12T14:33:02.505 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:bf5fcbac:::obj-fill-287:head# 2020-09-12T14:33:02.506 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:bf806d14:::obj-fill-115:head# 2020-09-12T14:33:02.506 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:bfb28823:::obj-fill-72:head# 2020-09-12T14:33:02.507 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:bfef805e:::obj-fill-119:head# 2020-09-12T14:33:02.507 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c02f8e10:::obj-fill-497:head# 2020-09-12T14:33:02.507 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c050cd6e:::obj-fill-8:head# 2020-09-12T14:33:02.508 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c218e297:::obj-fill-173:head# 2020-09-12T14:33:02.508 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c2bc66fe:::obj-fill-252:head# 2020-09-12T14:33:02.508 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c3b55903:::obj-fill-428:head# 2020-09-12T14:33:02.509 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c51907b0:::obj-fill-457:head# 2020-09-12T14:33:02.509 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c561dc83:::obj-fill-75:head# 2020-09-12T14:33:02.510 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c5d1e1a9:::obj-fill-268:head# 2020-09-12T14:33:02.510 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c60ab0d0:::obj-fill-303:head# 2020-09-12T14:33:02.510 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c644ce52:::obj-fill-514:head# 2020-09-12T14:33:02.510 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c6ab3a11:::obj-fill-407:head# 2020-09-12T14:33:02.511 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c74377bf:::obj-fill-432:head# 2020-09-12T14:33:02.511 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c767fff6:::obj-fill-531:head# 2020-09-12T14:33:02.512 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c7a9adad:::obj-fill-475:head# 2020-09-12T14:33:02.512 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c7e3a9c2:::obj-fill-465:head# 2020-09-12T14:33:02.513 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c850aed7:::obj-fill-424:head# 2020-09-12T14:33:02.513 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c8a0088a:::obj-fill-329:head# 2020-09-12T14:33:02.514 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:c9d419f9:::obj-fill-176:head# 2020-09-12T14:33:02.514 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:caf25ec2:::obj-fill-274:head# 2020-09-12T14:33:02.514 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:cb625830:::obj-fill-70:head# 2020-09-12T14:33:02.515 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:cbb65125:::obj-fill-529:head# 2020-09-12T14:33:02.515 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:cbe2544c:::obj-fill-3:head# 2020-09-12T14:33:02.516 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:cbfce4ee:::obj-fill-455:head# 2020-09-12T14:33:02.516 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:cc62419c:::obj-fill-498:head# 2020-09-12T14:33:02.517 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:cce831fa:::obj-fill-160:head# 2020-09-12T14:33:02.517 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:cda71af4:::obj-fill-122:head# 2020-09-12T14:33:02.517 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:cea7a728:::obj-fill-265:head# 2020-09-12T14:33:02.517 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:cee19f0a:::obj-fill-142:head# 2020-09-12T14:33:02.518 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:cf22117e:::obj-fill-499:head# 2020-09-12T14:33:02.518 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:cfbfa66d:::obj-fill-66:head# 2020-09-12T14:33:02.518 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:cfc2552c:::obj-fill-211:head# 2020-09-12T14:33:02.519 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:cfc563e8:::obj-fill-392:head# 2020-09-12T14:33:02.519 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d02d8880:::obj-fill-362:head# 2020-09-12T14:33:02.520 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d054947e:::obj-fill-67:head# 2020-09-12T14:33:02.520 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d061c977:::obj-fill-164:head# 2020-09-12T14:33:02.520 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d06a5f57:::obj-fill-296:head# 2020-09-12T14:33:02.521 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d110062a:::obj-fill-343:head# 2020-09-12T14:33:02.521 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d1b8ad47:::obj-fill-383:head# 2020-09-12T14:33:02.522 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d1e80db6:::obj-fill-117:head# 2020-09-12T14:33:02.522 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d210027e:::obj-fill-231:head# 2020-09-12T14:33:02.522 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d24428ed:::obj-fill-199:head# 2020-09-12T14:33:02.523 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d27c3c4b:::obj-fill-154:head# 2020-09-12T14:33:02.523 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d28957e3:::obj-fill-263:head# 2020-09-12T14:33:02.523 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d2d44da7:::obj-fill-544:head# 2020-09-12T14:33:02.524 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d31004a4:::obj-fill-434:head# 2020-09-12T14:33:02.524 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d32487c3:::obj-fill-385:head# 2020-09-12T14:33:02.524 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d340891c:::obj-fill-12:head# 2020-09-12T14:33:02.525 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d4135728:::obj-fill-310:head# 2020-09-12T14:33:02.525 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d5a43972:::obj-fill-9:head# 2020-09-12T14:33:02.525 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d5efeb15:::obj-fill-347:head# 2020-09-12T14:33:02.526 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d6463f70:::obj-fill-536:head# 2020-09-12T14:33:02.526 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d666d9b7:::obj-fill-420:head# 2020-09-12T14:33:02.526 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d6e86688:::obj-fill-166:head# 2020-09-12T14:33:02.526 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d730370d:::obj-fill-206:head# 2020-09-12T14:33:02.527 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d783551c:::obj-fill-109:head# 2020-09-12T14:33:02.527 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d7f77e5f:::obj-fill-551:head# 2020-09-12T14:33:02.527 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d843056c:::obj-fill-291:head# 2020-09-12T14:33:02.528 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d84eb173:::obj-fill-221:head# 2020-09-12T14:33:02.528 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d8dd069c:::obj-fill-60:head# 2020-09-12T14:33:02.528 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d8f2f750:::obj-fill-562:head# 2020-09-12T14:33:02.528 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d8f80ac2:::obj-fill-179:head# 2020-09-12T14:33:02.529 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:d9f904cf:::obj-fill-308:head# 2020-09-12T14:33:02.529 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:db0a534b:::obj-fill-410:head# 2020-09-12T14:33:02.530 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:dc4403f4:::obj-fill-416:head# 2020-09-12T14:33:02.530 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:dc66c3ca:::obj-fill-220:head# 2020-09-12T14:33:02.530 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:dc8198dc:::obj-fill-596:head# 2020-09-12T14:33:02.531 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:dcbade2b:::obj-fill-10:head# 2020-09-12T14:33:02.531 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ddb787ea:::obj-fill-556:head# 2020-09-12T14:33:02.531 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:de58065b:::obj-fill-306:head# 2020-09-12T14:33:02.532 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:deaa6fab:::obj-fill-194:head# 2020-09-12T14:33:02.532 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:dfee7714:::obj-fill-202:head# 2020-09-12T14:33:02.533 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:dff5df4a:::obj-fill-565:head# 2020-09-12T14:33:02.533 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:e2053442:::obj-fill-107:head# 2020-09-12T14:33:02.533 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:e32055b8:::obj-fill-57:head# 2020-09-12T14:33:02.534 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:e37fac1e:::obj-fill-161:head# 2020-09-12T14:33:02.534 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:e388f5aa:::obj-fill-273:head# 2020-09-12T14:33:02.534 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:e38b1f4d:::obj-fill-393:head# 2020-09-12T14:33:02.535 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:e4792474:::obj-fill-158:head# 2020-09-12T14:33:02.535 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:e4899cb0:::obj-fill-270:head# 2020-09-12T14:33:02.535 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:e62ae9f1:::obj-fill-288:head# 2020-09-12T14:33:02.536 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:e64fd95a:::obj-fill-449:head# 2020-09-12T14:33:02.536 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:e6682cfc:::obj-fill-373:head# 2020-09-12T14:33:02.536 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:e716717b:::obj-fill-178:head# 2020-09-12T14:33:02.537 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:e80226af:::obj-fill-354:head# 2020-09-12T14:33:02.537 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:e83a5940:::obj-fill-564:head# 2020-09-12T14:33:02.538 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:e854bfbe:::obj-fill-589:head# 2020-09-12T14:33:02.538 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:e8b197bc:::obj-fill-89:head# 2020-09-12T14:33:02.538 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:e92fca88:::obj-fill-397:head# 2020-09-12T14:33:02.539 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:e94d8f7b:::obj-fill-261:head# 2020-09-12T14:33:02.539 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:e98cc0a4:::obj-fill-525:head# 2020-09-12T14:33:02.539 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ea5a9459:::obj-fill-431:head# 2020-09-12T14:33:02.540 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ebc54d47:::obj-fill-258:head# 2020-09-12T14:33:02.540 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ec0002dd:::obj-fill-205:head# 2020-09-12T14:33:02.541 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ec740b5c:::obj-fill-46:head# 2020-09-12T14:33:02.541 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ed35dda4:::obj-fill-170:head# 2020-09-12T14:33:02.542 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ed51b8f7:::obj-fill-91:head# 2020-09-12T14:33:02.542 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:edd96161:::obj-fill-48:head# 2020-09-12T14:33:02.542 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ee3f0d4b:::obj-fill-47:head# 2020-09-12T14:33:02.543 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ee4784dd:::obj-fill-480:head# 2020-09-12T14:33:02.543 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ee5a8271:::obj-fill-198:head# 2020-09-12T14:33:02.544 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ef745b66:::obj-fill-145:head# 2020-09-12T14:33:02.544 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:efa5309d:::obj-fill-215:head# 2020-09-12T14:33:02.544 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:efbdd3a1:::obj-fill-276:head# 2020-09-12T14:33:02.545 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:f059d4c0:::obj-fill-482:head# 2020-09-12T14:33:02.545 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:f0f9fb7a:::obj-fill-379:head# 2020-09-12T14:33:02.545 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:f16a3656:::obj-fill-462:head# 2020-09-12T14:33:02.546 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:f1b395ae:::obj-fill-25:head# 2020-09-12T14:33:02.546 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:f5876dad:::obj-fill-359:head# 2020-09-12T14:33:02.547 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:f5a283cd:::obj-fill-534:head# 2020-09-12T14:33:02.547 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:f5cd905a:::obj-fill-358:head# 2020-09-12T14:33:02.547 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:f5e48302:::obj-fill-406:head# 2020-09-12T14:33:02.548 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:f5ea65fc:::obj-fill-52:head# 2020-09-12T14:33:02.548 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:f642de6b:::obj-fill-204:head# 2020-09-12T14:33:02.549 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:f716eafd:::obj-fill-502:head# 2020-09-12T14:33:02.549 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:f752e9cb:::obj-fill-23:head# 2020-09-12T14:33:02.549 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:f85fe42c:::obj-fill-346:head# 2020-09-12T14:33:02.550 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:f882fd3b:::obj-fill-355:head# 2020-09-12T14:33:02.550 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:fa133332:::obj-fill-309:head# 2020-09-12T14:33:02.551 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:fa875e52:::obj-fill-64:head# 2020-09-12T14:33:02.551 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:fad91a22:::obj-fill-481:head# 2020-09-12T14:33:02.551 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:fbbc3122:::obj-fill-111:head# 2020-09-12T14:33:02.552 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:fc539ddb:::obj-fill-181:head# 2020-09-12T14:33:02.552 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:fc54128d:::obj-fill-104:head# 2020-09-12T14:33:02.553 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:fe00aded:::obj-fill-365:head# 2020-09-12T14:33:02.553 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:fe1e9971:::obj-fill-189:head# 2020-09-12T14:33:02.554 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ff0e0563:::obj-fill-508:head# 2020-09-12T14:33:02.554 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ff4e156c:::obj-fill-441:head# 2020-09-12T14:33:02.554 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:ffa355f7:::obj-fill-500:head# 2020-09-12T14:33:02.555 INFO:tasks.workunit.client.0.smithi027.stderr:Read #1:fff6c93b:::obj-fill-277:head# 2020-09-12T14:33:02.555 INFO:tasks.workunit.client.0.smithi027.stderr:Export successful 2020-09-12T14:33:02.555 INFO:tasks.workunit.client.0.smithi027.stdout: marking collection for removal 2020-09-12T14:33:02.555 INFO:tasks.workunit.client.0.smithi027.stdout:setting '_remove' omap key 2020-09-12T14:33:02.556 INFO:tasks.workunit.client.0.smithi027.stdout:finish_remove_pgs 1.0_head removing 1.0 2020-09-12T14:33:03.641 INFO:tasks.workunit.client.0.smithi027.stderr:Remove successful 2020-09-12T14:33:04.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:477: TEST_backfill_multi_partial: activate_osd td/osd-backfill-space 1 2020-09-12T14:33:04.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/osd-backfill-space 2020-09-12T14:33:04.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T14:33:04.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=1 2020-09-12T14:33:04.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T14:33:04.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/osd-backfill-space/1 2020-09-12T14:33:04.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:33:04.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:33:04.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:33:04.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:33:04.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/osd-backfill-space/1' 2020-09-12T14:33:04.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/1/journal' 2020-09-12T14:33:04.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T14:33:04.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T14:33:04.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:33:04.075 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T14:33:04.075 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:33:04.076 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:33:04.076 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:33:04.076 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:33:04.076 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:33:04.078 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:33:04.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:33:04.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:33:04.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:33:04.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:33:04.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:33:04.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:33:04.079 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:33:04.079 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T14:33:04.079 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T14:33:04.079 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/osd-backfill-space/1 2020-09-12T14:33:04.079 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.1 2020-09-12T14:33:04.079 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 1 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/1 --osd-journal=td/osd-backfill-space/1/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:33:04.079 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/osd-backfill-space/1/whoami 2020-09-12T14:33:04.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 1 = 1 ']' 2020-09-12T14:33:04.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T14:33:04.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T14:33:04.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T14:33:04.080 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T14:33:04.104 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:33:04.103+0000 7f23cd386ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:33:04.106 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:33:04.105+0000 7f23cd386ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:33:04.108 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:33:04.107+0000 7f23cd386ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:33:04.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: wait_for_osd up 1 2020-09-12T14:33:04.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:33:04.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:33:04.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:33:04.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:33:04.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:33:04.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:33:04.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:33:04.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:33:04.504 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:33:04.631 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:33:04.630+0000 7f23cd386ec0 -1 Falling back to public interface 2020-09-12T14:33:04.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:33:05.655 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:33:05.653+0000 7f23cd386ec0 -1 osd.1 34 log_to_monitors {default=true} 2020-09-12T14:33:05.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:33:05.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:33:05.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:33:05.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:33:05.939 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:33:05.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:33:06.356 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up out weight 0 up_from 39 up_thru 33 down_at 35 last_clean_interval [10,34) [v2:127.0.0.1:6810/194660,v1:127.0.0.1:6811/194660] [v2:127.0.0.1:6812/194660,v1:127.0.0.1:6813/194660] exists,up ce656e6a-6430-4cc2-bd95-29130ea1369f 2020-09-12T14:33:06.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:33:06.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:33:06.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:33:06.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:479: TEST_backfill_multi_partial: ceph pg dump pgs 2020-09-12T14:33:06.978 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:33:06.979 INFO:tasks.workunit.client.0.smithi027.stdout:3.0 0 0 0 0 0 0 0 0 0 0 active+clean 2020-09-12T14:33:02.061981+0000 0'0 38:29 [2,0] 2 [2,0] 2 0'0 2020-09-12T14:32:03.946070+0000 0'0 2020-09-12T14:32:03.946070+0000 0 2020-09-12T14:33:06.979 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 542 0 0 0 0 1415162 0 0 42 42 stale+active+clean 2020-09-12T14:32:01.131349+0000 34'542 34:569 [1] 1 [1] 1 0'0 2020-09-12T14:31:58.829524+0000 0'0 2020-09-12T14:31:58.829524+0000 0 2020-09-12T14:33:06.979 INFO:tasks.workunit.client.0.smithi027.stdout:2.0 0 0 0 0 0 0 0 0 0 0 active+clean 2020-09-12T14:33:02.062235+0000 0'0 38:41 [2,0] 2 [2,0] 2 0'0 2020-09-12T14:32:01.404494+0000 0'0 2020-09-12T14:32:01.404494+0000 0 2020-09-12T14:33:06.979 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:33:06.979 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:33:06.980 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:33:06.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:481: TEST_backfill_multi_partial: dd if=/dev/urandom of=td/osd-backfill-space/datafile bs=1024 count=1 2020-09-12T14:33:06.990 INFO:tasks.workunit.client.0.smithi027.stderr:1+0 records in 2020-09-12T14:33:06.991 INFO:tasks.workunit.client.0.smithi027.stderr:1+0 records out 2020-09-12T14:33:06.991 INFO:tasks.workunit.client.0.smithi027.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 8.1213e-05 s, 12.6 MB/s 2020-09-12T14:33:06.991 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: seq 1 600 2020-09-12T14:33:06.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:06.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-1 td/osd-backfill-space/datafile 2020-09-12T14:33:07.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:07.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-2 td/osd-backfill-space/datafile 2020-09-12T14:33:07.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:07.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-3 td/osd-backfill-space/datafile 2020-09-12T14:33:07.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:07.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-4 td/osd-backfill-space/datafile 2020-09-12T14:33:07.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:07.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-5 td/osd-backfill-space/datafile 2020-09-12T14:33:07.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:07.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-6 td/osd-backfill-space/datafile 2020-09-12T14:33:07.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:07.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-7 td/osd-backfill-space/datafile 2020-09-12T14:33:07.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:07.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-8 td/osd-backfill-space/datafile 2020-09-12T14:33:08.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:08.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-9 td/osd-backfill-space/datafile 2020-09-12T14:33:08.122 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:08.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-10 td/osd-backfill-space/datafile 2020-09-12T14:33:08.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:08.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-11 td/osd-backfill-space/datafile 2020-09-12T14:33:08.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:08.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-12 td/osd-backfill-space/datafile 2020-09-12T14:33:08.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:08.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-13 td/osd-backfill-space/datafile 2020-09-12T14:33:08.667 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:08.667 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-14 td/osd-backfill-space/datafile 2020-09-12T14:33:08.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:08.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-15 td/osd-backfill-space/datafile 2020-09-12T14:33:08.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:08.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-16 td/osd-backfill-space/datafile 2020-09-12T14:33:09.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:09.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-17 td/osd-backfill-space/datafile 2020-09-12T14:33:09.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:09.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-18 td/osd-backfill-space/datafile 2020-09-12T14:33:09.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:09.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-19 td/osd-backfill-space/datafile 2020-09-12T14:33:09.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:09.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-20 td/osd-backfill-space/datafile 2020-09-12T14:33:09.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:09.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-21 td/osd-backfill-space/datafile 2020-09-12T14:33:09.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:09.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-22 td/osd-backfill-space/datafile 2020-09-12T14:33:09.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:09.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-23 td/osd-backfill-space/datafile 2020-09-12T14:33:09.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:09.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-24 td/osd-backfill-space/datafile 2020-09-12T14:33:10.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:10.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-25 td/osd-backfill-space/datafile 2020-09-12T14:33:10.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:10.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-26 td/osd-backfill-space/datafile 2020-09-12T14:33:10.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:10.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-27 td/osd-backfill-space/datafile 2020-09-12T14:33:10.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:10.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-28 td/osd-backfill-space/datafile 2020-09-12T14:33:10.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:10.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-29 td/osd-backfill-space/datafile 2020-09-12T14:33:10.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:10.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-30 td/osd-backfill-space/datafile 2020-09-12T14:33:10.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:10.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-31 td/osd-backfill-space/datafile 2020-09-12T14:33:10.894 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:10.894 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-32 td/osd-backfill-space/datafile 2020-09-12T14:33:11.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:11.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-33 td/osd-backfill-space/datafile 2020-09-12T14:33:11.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:11.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-34 td/osd-backfill-space/datafile 2020-09-12T14:33:11.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:11.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-35 td/osd-backfill-space/datafile 2020-09-12T14:33:11.422 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:11.422 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-36 td/osd-backfill-space/datafile 2020-09-12T14:33:11.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:11.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-37 td/osd-backfill-space/datafile 2020-09-12T14:33:11.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:11.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-38 td/osd-backfill-space/datafile 2020-09-12T14:33:11.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:11.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-39 td/osd-backfill-space/datafile 2020-09-12T14:33:11.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:11.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-40 td/osd-backfill-space/datafile 2020-09-12T14:33:12.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:12.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-41 td/osd-backfill-space/datafile 2020-09-12T14:33:12.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:12.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-42 td/osd-backfill-space/datafile 2020-09-12T14:33:12.418 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:12.418 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-43 td/osd-backfill-space/datafile 2020-09-12T14:33:12.552 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:12.552 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-44 td/osd-backfill-space/datafile 2020-09-12T14:33:12.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:12.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-45 td/osd-backfill-space/datafile 2020-09-12T14:33:12.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:12.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-46 td/osd-backfill-space/datafile 2020-09-12T14:33:12.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:12.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-47 td/osd-backfill-space/datafile 2020-09-12T14:33:13.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:13.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-48 td/osd-backfill-space/datafile 2020-09-12T14:33:13.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:13.231 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-49 td/osd-backfill-space/datafile 2020-09-12T14:33:13.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:13.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-50 td/osd-backfill-space/datafile 2020-09-12T14:33:13.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:13.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-51 td/osd-backfill-space/datafile 2020-09-12T14:33:13.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:13.616 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-52 td/osd-backfill-space/datafile 2020-09-12T14:33:13.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:13.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-53 td/osd-backfill-space/datafile 2020-09-12T14:33:13.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:13.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-54 td/osd-backfill-space/datafile 2020-09-12T14:33:13.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:13.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-55 td/osd-backfill-space/datafile 2020-09-12T14:33:14.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:14.110 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-56 td/osd-backfill-space/datafile 2020-09-12T14:33:14.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:14.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-57 td/osd-backfill-space/datafile 2020-09-12T14:33:14.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:14.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-58 td/osd-backfill-space/datafile 2020-09-12T14:33:14.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:14.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-59 td/osd-backfill-space/datafile 2020-09-12T14:33:14.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:14.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-60 td/osd-backfill-space/datafile 2020-09-12T14:33:14.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:14.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-61 td/osd-backfill-space/datafile 2020-09-12T14:33:14.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:14.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-62 td/osd-backfill-space/datafile 2020-09-12T14:33:15.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:15.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-63 td/osd-backfill-space/datafile 2020-09-12T14:33:15.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:15.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-64 td/osd-backfill-space/datafile 2020-09-12T14:33:15.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:15.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-65 td/osd-backfill-space/datafile 2020-09-12T14:33:15.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:15.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-66 td/osd-backfill-space/datafile 2020-09-12T14:33:15.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:15.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-67 td/osd-backfill-space/datafile 2020-09-12T14:33:15.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:15.659 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-68 td/osd-backfill-space/datafile 2020-09-12T14:33:15.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:15.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-69 td/osd-backfill-space/datafile 2020-09-12T14:33:15.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:15.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-70 td/osd-backfill-space/datafile 2020-09-12T14:33:16.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:16.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-71 td/osd-backfill-space/datafile 2020-09-12T14:33:16.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:16.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-72 td/osd-backfill-space/datafile 2020-09-12T14:33:16.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:16.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-73 td/osd-backfill-space/datafile 2020-09-12T14:33:16.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:16.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-74 td/osd-backfill-space/datafile 2020-09-12T14:33:16.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:16.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-75 td/osd-backfill-space/datafile 2020-09-12T14:33:16.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:16.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-76 td/osd-backfill-space/datafile 2020-09-12T14:33:16.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:16.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-77 td/osd-backfill-space/datafile 2020-09-12T14:33:16.915 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:16.915 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-78 td/osd-backfill-space/datafile 2020-09-12T14:33:17.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:17.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-79 td/osd-backfill-space/datafile 2020-09-12T14:33:17.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:17.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-80 td/osd-backfill-space/datafile 2020-09-12T14:33:17.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:17.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-81 td/osd-backfill-space/datafile 2020-09-12T14:33:17.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:17.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-82 td/osd-backfill-space/datafile 2020-09-12T14:33:17.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:17.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-83 td/osd-backfill-space/datafile 2020-09-12T14:33:17.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:17.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-84 td/osd-backfill-space/datafile 2020-09-12T14:33:17.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:17.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-85 td/osd-backfill-space/datafile 2020-09-12T14:33:17.928 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:17.928 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-86 td/osd-backfill-space/datafile 2020-09-12T14:33:18.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:18.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-87 td/osd-backfill-space/datafile 2020-09-12T14:33:18.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:18.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-88 td/osd-backfill-space/datafile 2020-09-12T14:33:18.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:18.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-89 td/osd-backfill-space/datafile 2020-09-12T14:33:18.431 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:18.431 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-90 td/osd-backfill-space/datafile 2020-09-12T14:33:18.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:18.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-91 td/osd-backfill-space/datafile 2020-09-12T14:33:18.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:18.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-92 td/osd-backfill-space/datafile 2020-09-12T14:33:18.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:18.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-93 td/osd-backfill-space/datafile 2020-09-12T14:33:18.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:18.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-94 td/osd-backfill-space/datafile 2020-09-12T14:33:19.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:19.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-95 td/osd-backfill-space/datafile 2020-09-12T14:33:19.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:19.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-96 td/osd-backfill-space/datafile 2020-09-12T14:33:19.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:19.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-97 td/osd-backfill-space/datafile 2020-09-12T14:33:19.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:19.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-98 td/osd-backfill-space/datafile 2020-09-12T14:33:19.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:19.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-99 td/osd-backfill-space/datafile 2020-09-12T14:33:19.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:19.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-100 td/osd-backfill-space/datafile 2020-09-12T14:33:19.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:19.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-101 td/osd-backfill-space/datafile 2020-09-12T14:33:19.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:19.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-102 td/osd-backfill-space/datafile 2020-09-12T14:33:20.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:20.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-103 td/osd-backfill-space/datafile 2020-09-12T14:33:20.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:20.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-104 td/osd-backfill-space/datafile 2020-09-12T14:33:20.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:20.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-105 td/osd-backfill-space/datafile 2020-09-12T14:33:20.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:20.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-106 td/osd-backfill-space/datafile 2020-09-12T14:33:20.550 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:20.550 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-107 td/osd-backfill-space/datafile 2020-09-12T14:33:20.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:20.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-108 td/osd-backfill-space/datafile 2020-09-12T14:33:20.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:20.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-109 td/osd-backfill-space/datafile 2020-09-12T14:33:20.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:20.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-110 td/osd-backfill-space/datafile 2020-09-12T14:33:21.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:21.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-111 td/osd-backfill-space/datafile 2020-09-12T14:33:21.178 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:21.178 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-112 td/osd-backfill-space/datafile 2020-09-12T14:33:21.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:21.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-113 td/osd-backfill-space/datafile 2020-09-12T14:33:21.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:21.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-114 td/osd-backfill-space/datafile 2020-09-12T14:33:21.547 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:21.547 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-115 td/osd-backfill-space/datafile 2020-09-12T14:33:21.664 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:21.664 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-116 td/osd-backfill-space/datafile 2020-09-12T14:33:21.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:21.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-117 td/osd-backfill-space/datafile 2020-09-12T14:33:21.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:21.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-118 td/osd-backfill-space/datafile 2020-09-12T14:33:22.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:22.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-119 td/osd-backfill-space/datafile 2020-09-12T14:33:22.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:22.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-120 td/osd-backfill-space/datafile 2020-09-12T14:33:22.284 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:22.284 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-121 td/osd-backfill-space/datafile 2020-09-12T14:33:22.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:22.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-122 td/osd-backfill-space/datafile 2020-09-12T14:33:22.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:22.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-123 td/osd-backfill-space/datafile 2020-09-12T14:33:22.669 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:22.669 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-124 td/osd-backfill-space/datafile 2020-09-12T14:33:22.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:22.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-125 td/osd-backfill-space/datafile 2020-09-12T14:33:22.929 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:22.929 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-126 td/osd-backfill-space/datafile 2020-09-12T14:33:23.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:23.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-127 td/osd-backfill-space/datafile 2020-09-12T14:33:23.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:23.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-128 td/osd-backfill-space/datafile 2020-09-12T14:33:23.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:23.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-129 td/osd-backfill-space/datafile 2020-09-12T14:33:23.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:23.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-130 td/osd-backfill-space/datafile 2020-09-12T14:33:23.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:23.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-131 td/osd-backfill-space/datafile 2020-09-12T14:33:23.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:23.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-132 td/osd-backfill-space/datafile 2020-09-12T14:33:23.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:23.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-133 td/osd-backfill-space/datafile 2020-09-12T14:33:23.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:23.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-134 td/osd-backfill-space/datafile 2020-09-12T14:33:24.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:24.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-135 td/osd-backfill-space/datafile 2020-09-12T14:33:24.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:24.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-136 td/osd-backfill-space/datafile 2020-09-12T14:33:24.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:24.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-137 td/osd-backfill-space/datafile 2020-09-12T14:33:24.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:24.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-138 td/osd-backfill-space/datafile 2020-09-12T14:33:24.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:24.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-139 td/osd-backfill-space/datafile 2020-09-12T14:33:24.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:24.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-140 td/osd-backfill-space/datafile 2020-09-12T14:33:24.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:24.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-141 td/osd-backfill-space/datafile 2020-09-12T14:33:25.106 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:25.106 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-142 td/osd-backfill-space/datafile 2020-09-12T14:33:25.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:25.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-143 td/osd-backfill-space/datafile 2020-09-12T14:33:25.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:25.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-144 td/osd-backfill-space/datafile 2020-09-12T14:33:25.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:25.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-145 td/osd-backfill-space/datafile 2020-09-12T14:33:25.617 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:25.617 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-146 td/osd-backfill-space/datafile 2020-09-12T14:33:25.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:25.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-147 td/osd-backfill-space/datafile 2020-09-12T14:33:25.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:25.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-148 td/osd-backfill-space/datafile 2020-09-12T14:33:26.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:26.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-149 td/osd-backfill-space/datafile 2020-09-12T14:33:26.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:26.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-150 td/osd-backfill-space/datafile 2020-09-12T14:33:26.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:26.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-151 td/osd-backfill-space/datafile 2020-09-12T14:33:26.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:26.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-152 td/osd-backfill-space/datafile 2020-09-12T14:33:26.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:26.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-153 td/osd-backfill-space/datafile 2020-09-12T14:33:26.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:26.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-154 td/osd-backfill-space/datafile 2020-09-12T14:33:26.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:26.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-155 td/osd-backfill-space/datafile 2020-09-12T14:33:26.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:26.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-156 td/osd-backfill-space/datafile 2020-09-12T14:33:27.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:27.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-157 td/osd-backfill-space/datafile 2020-09-12T14:33:27.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:27.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-158 td/osd-backfill-space/datafile 2020-09-12T14:33:27.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:27.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-159 td/osd-backfill-space/datafile 2020-09-12T14:33:27.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:27.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-160 td/osd-backfill-space/datafile 2020-09-12T14:33:27.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:27.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-161 td/osd-backfill-space/datafile 2020-09-12T14:33:27.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:27.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-162 td/osd-backfill-space/datafile 2020-09-12T14:33:27.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:27.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-163 td/osd-backfill-space/datafile 2020-09-12T14:33:28.079 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:28.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-164 td/osd-backfill-space/datafile 2020-09-12T14:33:28.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:28.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-165 td/osd-backfill-space/datafile 2020-09-12T14:33:28.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:28.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-166 td/osd-backfill-space/datafile 2020-09-12T14:33:28.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:28.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-167 td/osd-backfill-space/datafile 2020-09-12T14:33:28.616 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:28.616 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-168 td/osd-backfill-space/datafile 2020-09-12T14:33:28.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:28.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-169 td/osd-backfill-space/datafile 2020-09-12T14:33:28.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:28.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-170 td/osd-backfill-space/datafile 2020-09-12T14:33:29.009 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:29.009 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-171 td/osd-backfill-space/datafile 2020-09-12T14:33:29.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:29.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-172 td/osd-backfill-space/datafile 2020-09-12T14:33:29.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:29.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-173 td/osd-backfill-space/datafile 2020-09-12T14:33:29.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:29.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-174 td/osd-backfill-space/datafile 2020-09-12T14:33:29.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:29.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-175 td/osd-backfill-space/datafile 2020-09-12T14:33:29.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:29.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-176 td/osd-backfill-space/datafile 2020-09-12T14:33:29.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:29.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-177 td/osd-backfill-space/datafile 2020-09-12T14:33:29.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:29.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-178 td/osd-backfill-space/datafile 2020-09-12T14:33:30.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:30.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-179 td/osd-backfill-space/datafile 2020-09-12T14:33:30.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:30.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-180 td/osd-backfill-space/datafile 2020-09-12T14:33:30.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:30.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-181 td/osd-backfill-space/datafile 2020-09-12T14:33:30.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:30.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-182 td/osd-backfill-space/datafile 2020-09-12T14:33:30.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:30.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-183 td/osd-backfill-space/datafile 2020-09-12T14:33:30.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:30.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-184 td/osd-backfill-space/datafile 2020-09-12T14:33:30.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:30.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-185 td/osd-backfill-space/datafile 2020-09-12T14:33:30.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:30.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-186 td/osd-backfill-space/datafile 2020-09-12T14:33:31.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:31.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-187 td/osd-backfill-space/datafile 2020-09-12T14:33:31.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:31.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-188 td/osd-backfill-space/datafile 2020-09-12T14:33:31.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:31.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-189 td/osd-backfill-space/datafile 2020-09-12T14:33:31.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:31.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-190 td/osd-backfill-space/datafile 2020-09-12T14:33:31.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:31.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-191 td/osd-backfill-space/datafile 2020-09-12T14:33:31.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:31.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-192 td/osd-backfill-space/datafile 2020-09-12T14:33:31.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:31.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-193 td/osd-backfill-space/datafile 2020-09-12T14:33:31.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:31.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-194 td/osd-backfill-space/datafile 2020-09-12T14:33:32.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:32.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-195 td/osd-backfill-space/datafile 2020-09-12T14:33:32.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:32.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-196 td/osd-backfill-space/datafile 2020-09-12T14:33:32.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:32.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-197 td/osd-backfill-space/datafile 2020-09-12T14:33:32.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:32.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-198 td/osd-backfill-space/datafile 2020-09-12T14:33:32.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:32.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-199 td/osd-backfill-space/datafile 2020-09-12T14:33:32.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:32.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-200 td/osd-backfill-space/datafile 2020-09-12T14:33:32.894 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:32.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-201 td/osd-backfill-space/datafile 2020-09-12T14:33:33.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:33.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-202 td/osd-backfill-space/datafile 2020-09-12T14:33:33.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:33.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-203 td/osd-backfill-space/datafile 2020-09-12T14:33:33.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:33.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-204 td/osd-backfill-space/datafile 2020-09-12T14:33:33.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:33.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-205 td/osd-backfill-space/datafile 2020-09-12T14:33:33.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:33.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-206 td/osd-backfill-space/datafile 2020-09-12T14:33:33.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:33.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-207 td/osd-backfill-space/datafile 2020-09-12T14:33:33.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:33.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-208 td/osd-backfill-space/datafile 2020-09-12T14:33:33.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:33.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-209 td/osd-backfill-space/datafile 2020-09-12T14:33:33.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:33.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-210 td/osd-backfill-space/datafile 2020-09-12T14:33:34.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:34.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-211 td/osd-backfill-space/datafile 2020-09-12T14:33:34.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:34.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-212 td/osd-backfill-space/datafile 2020-09-12T14:33:34.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:34.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-213 td/osd-backfill-space/datafile 2020-09-12T14:33:34.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:34.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-214 td/osd-backfill-space/datafile 2020-09-12T14:33:34.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:34.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-215 td/osd-backfill-space/datafile 2020-09-12T14:33:34.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:34.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-216 td/osd-backfill-space/datafile 2020-09-12T14:33:34.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:34.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-217 td/osd-backfill-space/datafile 2020-09-12T14:33:34.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:34.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-218 td/osd-backfill-space/datafile 2020-09-12T14:33:35.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:35.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-219 td/osd-backfill-space/datafile 2020-09-12T14:33:35.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:35.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-220 td/osd-backfill-space/datafile 2020-09-12T14:33:35.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:35.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-221 td/osd-backfill-space/datafile 2020-09-12T14:33:35.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:35.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-222 td/osd-backfill-space/datafile 2020-09-12T14:33:35.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:35.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-223 td/osd-backfill-space/datafile 2020-09-12T14:33:35.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:35.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-224 td/osd-backfill-space/datafile 2020-09-12T14:33:35.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:35.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-225 td/osd-backfill-space/datafile 2020-09-12T14:33:35.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:35.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-226 td/osd-backfill-space/datafile 2020-09-12T14:33:36.085 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:36.086 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-227 td/osd-backfill-space/datafile 2020-09-12T14:33:36.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:36.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-228 td/osd-backfill-space/datafile 2020-09-12T14:33:36.354 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:36.354 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-229 td/osd-backfill-space/datafile 2020-09-12T14:33:36.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:36.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-230 td/osd-backfill-space/datafile 2020-09-12T14:33:36.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:36.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-231 td/osd-backfill-space/datafile 2020-09-12T14:33:36.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:36.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-232 td/osd-backfill-space/datafile 2020-09-12T14:33:36.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:36.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-233 td/osd-backfill-space/datafile 2020-09-12T14:33:36.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:36.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-234 td/osd-backfill-space/datafile 2020-09-12T14:33:37.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:37.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-235 td/osd-backfill-space/datafile 2020-09-12T14:33:37.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:37.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-236 td/osd-backfill-space/datafile 2020-09-12T14:33:37.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:37.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-237 td/osd-backfill-space/datafile 2020-09-12T14:33:37.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:37.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-238 td/osd-backfill-space/datafile 2020-09-12T14:33:37.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:37.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-239 td/osd-backfill-space/datafile 2020-09-12T14:33:37.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:37.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-240 td/osd-backfill-space/datafile 2020-09-12T14:33:38.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:38.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-241 td/osd-backfill-space/datafile 2020-09-12T14:33:38.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:38.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-242 td/osd-backfill-space/datafile 2020-09-12T14:33:38.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:38.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-243 td/osd-backfill-space/datafile 2020-09-12T14:33:38.466 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:38.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-244 td/osd-backfill-space/datafile 2020-09-12T14:33:38.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:38.597 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-245 td/osd-backfill-space/datafile 2020-09-12T14:33:38.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:38.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-246 td/osd-backfill-space/datafile 2020-09-12T14:33:38.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:38.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-247 td/osd-backfill-space/datafile 2020-09-12T14:33:38.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:38.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-248 td/osd-backfill-space/datafile 2020-09-12T14:33:39.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:39.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-249 td/osd-backfill-space/datafile 2020-09-12T14:33:39.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:39.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-250 td/osd-backfill-space/datafile 2020-09-12T14:33:39.362 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:39.362 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-251 td/osd-backfill-space/datafile 2020-09-12T14:33:39.496 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:39.496 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-252 td/osd-backfill-space/datafile 2020-09-12T14:33:39.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:39.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-253 td/osd-backfill-space/datafile 2020-09-12T14:33:39.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:39.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-254 td/osd-backfill-space/datafile 2020-09-12T14:33:39.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:39.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-255 td/osd-backfill-space/datafile 2020-09-12T14:33:40.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:40.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-256 td/osd-backfill-space/datafile 2020-09-12T14:33:40.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:40.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-257 td/osd-backfill-space/datafile 2020-09-12T14:33:40.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:40.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-258 td/osd-backfill-space/datafile 2020-09-12T14:33:40.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:40.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-259 td/osd-backfill-space/datafile 2020-09-12T14:33:40.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:40.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-260 td/osd-backfill-space/datafile 2020-09-12T14:33:40.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:40.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-261 td/osd-backfill-space/datafile 2020-09-12T14:33:40.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:40.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-262 td/osd-backfill-space/datafile 2020-09-12T14:33:40.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:40.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-263 td/osd-backfill-space/datafile 2020-09-12T14:33:41.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:41.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-264 td/osd-backfill-space/datafile 2020-09-12T14:33:41.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:41.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-265 td/osd-backfill-space/datafile 2020-09-12T14:33:41.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:41.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-266 td/osd-backfill-space/datafile 2020-09-12T14:33:41.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:41.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-267 td/osd-backfill-space/datafile 2020-09-12T14:33:41.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:41.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-268 td/osd-backfill-space/datafile 2020-09-12T14:33:41.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:41.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-269 td/osd-backfill-space/datafile 2020-09-12T14:33:41.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:41.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-270 td/osd-backfill-space/datafile 2020-09-12T14:33:41.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:41.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-271 td/osd-backfill-space/datafile 2020-09-12T14:33:42.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:42.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-272 td/osd-backfill-space/datafile 2020-09-12T14:33:42.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:42.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-273 td/osd-backfill-space/datafile 2020-09-12T14:33:42.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:42.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-274 td/osd-backfill-space/datafile 2020-09-12T14:33:42.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:42.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-275 td/osd-backfill-space/datafile 2020-09-12T14:33:42.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:42.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-276 td/osd-backfill-space/datafile 2020-09-12T14:33:42.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:42.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-277 td/osd-backfill-space/datafile 2020-09-12T14:33:42.862 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:42.862 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-278 td/osd-backfill-space/datafile 2020-09-12T14:33:42.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:42.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-279 td/osd-backfill-space/datafile 2020-09-12T14:33:43.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:43.122 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-280 td/osd-backfill-space/datafile 2020-09-12T14:33:43.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:43.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-281 td/osd-backfill-space/datafile 2020-09-12T14:33:43.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:43.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-282 td/osd-backfill-space/datafile 2020-09-12T14:33:43.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:43.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-283 td/osd-backfill-space/datafile 2020-09-12T14:33:43.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:43.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-284 td/osd-backfill-space/datafile 2020-09-12T14:33:43.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:43.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-285 td/osd-backfill-space/datafile 2020-09-12T14:33:43.933 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:43.934 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-286 td/osd-backfill-space/datafile 2020-09-12T14:33:44.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:44.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-287 td/osd-backfill-space/datafile 2020-09-12T14:33:44.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:44.177 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-288 td/osd-backfill-space/datafile 2020-09-12T14:33:44.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:44.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-289 td/osd-backfill-space/datafile 2020-09-12T14:33:44.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:44.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-290 td/osd-backfill-space/datafile 2020-09-12T14:33:44.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:44.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-291 td/osd-backfill-space/datafile 2020-09-12T14:33:44.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:44.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-292 td/osd-backfill-space/datafile 2020-09-12T14:33:44.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:44.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-293 td/osd-backfill-space/datafile 2020-09-12T14:33:44.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:44.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-294 td/osd-backfill-space/datafile 2020-09-12T14:33:45.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:45.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-295 td/osd-backfill-space/datafile 2020-09-12T14:33:45.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:45.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-296 td/osd-backfill-space/datafile 2020-09-12T14:33:45.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:45.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-297 td/osd-backfill-space/datafile 2020-09-12T14:33:45.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:45.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-298 td/osd-backfill-space/datafile 2020-09-12T14:33:45.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:45.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-299 td/osd-backfill-space/datafile 2020-09-12T14:33:45.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:45.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-300 td/osd-backfill-space/datafile 2020-09-12T14:33:45.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:45.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-301 td/osd-backfill-space/datafile 2020-09-12T14:33:45.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:45.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-302 td/osd-backfill-space/datafile 2020-09-12T14:33:46.086 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:46.086 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-303 td/osd-backfill-space/datafile 2020-09-12T14:33:46.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:46.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-304 td/osd-backfill-space/datafile 2020-09-12T14:33:46.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:46.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-305 td/osd-backfill-space/datafile 2020-09-12T14:33:46.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:46.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-306 td/osd-backfill-space/datafile 2020-09-12T14:33:46.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:46.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-307 td/osd-backfill-space/datafile 2020-09-12T14:33:46.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:46.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-308 td/osd-backfill-space/datafile 2020-09-12T14:33:46.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:46.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-309 td/osd-backfill-space/datafile 2020-09-12T14:33:46.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:46.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-310 td/osd-backfill-space/datafile 2020-09-12T14:33:47.091 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:47.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-311 td/osd-backfill-space/datafile 2020-09-12T14:33:47.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:47.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-312 td/osd-backfill-space/datafile 2020-09-12T14:33:47.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:47.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-313 td/osd-backfill-space/datafile 2020-09-12T14:33:47.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:47.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-314 td/osd-backfill-space/datafile 2020-09-12T14:33:47.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:47.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-315 td/osd-backfill-space/datafile 2020-09-12T14:33:47.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:47.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-316 td/osd-backfill-space/datafile 2020-09-12T14:33:47.862 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:47.862 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-317 td/osd-backfill-space/datafile 2020-09-12T14:33:47.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:47.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-318 td/osd-backfill-space/datafile 2020-09-12T14:33:48.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:48.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-319 td/osd-backfill-space/datafile 2020-09-12T14:33:48.264 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:48.264 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-320 td/osd-backfill-space/datafile 2020-09-12T14:33:48.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:48.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-321 td/osd-backfill-space/datafile 2020-09-12T14:33:48.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:48.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-322 td/osd-backfill-space/datafile 2020-09-12T14:33:48.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:48.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-323 td/osd-backfill-space/datafile 2020-09-12T14:33:48.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:48.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-324 td/osd-backfill-space/datafile 2020-09-12T14:33:48.892 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:48.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-325 td/osd-backfill-space/datafile 2020-09-12T14:33:49.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:49.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-326 td/osd-backfill-space/datafile 2020-09-12T14:33:49.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:49.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-327 td/osd-backfill-space/datafile 2020-09-12T14:33:49.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:49.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-328 td/osd-backfill-space/datafile 2020-09-12T14:33:49.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:49.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-329 td/osd-backfill-space/datafile 2020-09-12T14:33:49.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:49.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-330 td/osd-backfill-space/datafile 2020-09-12T14:33:49.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:49.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-331 td/osd-backfill-space/datafile 2020-09-12T14:33:49.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:49.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-332 td/osd-backfill-space/datafile 2020-09-12T14:33:49.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:49.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-333 td/osd-backfill-space/datafile 2020-09-12T14:33:50.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:50.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-334 td/osd-backfill-space/datafile 2020-09-12T14:33:50.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:50.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-335 td/osd-backfill-space/datafile 2020-09-12T14:33:50.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:50.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-336 td/osd-backfill-space/datafile 2020-09-12T14:33:50.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:50.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-337 td/osd-backfill-space/datafile 2020-09-12T14:33:50.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:50.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-338 td/osd-backfill-space/datafile 2020-09-12T14:33:50.669 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:50.669 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-339 td/osd-backfill-space/datafile 2020-09-12T14:33:50.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:50.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-340 td/osd-backfill-space/datafile 2020-09-12T14:33:50.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:50.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-341 td/osd-backfill-space/datafile 2020-09-12T14:33:51.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:51.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-342 td/osd-backfill-space/datafile 2020-09-12T14:33:51.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:51.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-343 td/osd-backfill-space/datafile 2020-09-12T14:33:51.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:51.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-344 td/osd-backfill-space/datafile 2020-09-12T14:33:51.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:51.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-345 td/osd-backfill-space/datafile 2020-09-12T14:33:51.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:51.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-346 td/osd-backfill-space/datafile 2020-09-12T14:33:51.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:51.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-347 td/osd-backfill-space/datafile 2020-09-12T14:33:51.765 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:51.766 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-348 td/osd-backfill-space/datafile 2020-09-12T14:33:51.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:51.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-349 td/osd-backfill-space/datafile 2020-09-12T14:33:52.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:52.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-350 td/osd-backfill-space/datafile 2020-09-12T14:33:52.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:52.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-351 td/osd-backfill-space/datafile 2020-09-12T14:33:52.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:52.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-352 td/osd-backfill-space/datafile 2020-09-12T14:33:52.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:52.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-353 td/osd-backfill-space/datafile 2020-09-12T14:33:52.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:52.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-354 td/osd-backfill-space/datafile 2020-09-12T14:33:52.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:52.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-355 td/osd-backfill-space/datafile 2020-09-12T14:33:52.771 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:52.771 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-356 td/osd-backfill-space/datafile 2020-09-12T14:33:52.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:52.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-357 td/osd-backfill-space/datafile 2020-09-12T14:33:53.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:53.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-358 td/osd-backfill-space/datafile 2020-09-12T14:33:53.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:53.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-359 td/osd-backfill-space/datafile 2020-09-12T14:33:53.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:53.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-360 td/osd-backfill-space/datafile 2020-09-12T14:33:53.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:53.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-361 td/osd-backfill-space/datafile 2020-09-12T14:33:53.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:53.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-362 td/osd-backfill-space/datafile 2020-09-12T14:33:53.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:53.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-363 td/osd-backfill-space/datafile 2020-09-12T14:33:53.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:53.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-364 td/osd-backfill-space/datafile 2020-09-12T14:33:53.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:53.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-365 td/osd-backfill-space/datafile 2020-09-12T14:33:53.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:53.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-366 td/osd-backfill-space/datafile 2020-09-12T14:33:54.119 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:54.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-367 td/osd-backfill-space/datafile 2020-09-12T14:33:54.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:54.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-368 td/osd-backfill-space/datafile 2020-09-12T14:33:54.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:54.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-369 td/osd-backfill-space/datafile 2020-09-12T14:33:54.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:54.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-370 td/osd-backfill-space/datafile 2020-09-12T14:33:54.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:54.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-371 td/osd-backfill-space/datafile 2020-09-12T14:33:54.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:54.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-372 td/osd-backfill-space/datafile 2020-09-12T14:33:54.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:54.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-373 td/osd-backfill-space/datafile 2020-09-12T14:33:55.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:55.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-374 td/osd-backfill-space/datafile 2020-09-12T14:33:55.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:55.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-375 td/osd-backfill-space/datafile 2020-09-12T14:33:55.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:55.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-376 td/osd-backfill-space/datafile 2020-09-12T14:33:55.417 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:55.418 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-377 td/osd-backfill-space/datafile 2020-09-12T14:33:55.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:55.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-378 td/osd-backfill-space/datafile 2020-09-12T14:33:55.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:55.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-379 td/osd-backfill-space/datafile 2020-09-12T14:33:55.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:55.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-380 td/osd-backfill-space/datafile 2020-09-12T14:33:55.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:55.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-381 td/osd-backfill-space/datafile 2020-09-12T14:33:56.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:56.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-382 td/osd-backfill-space/datafile 2020-09-12T14:33:56.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:56.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-383 td/osd-backfill-space/datafile 2020-09-12T14:33:56.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:56.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-384 td/osd-backfill-space/datafile 2020-09-12T14:33:56.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:56.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-385 td/osd-backfill-space/datafile 2020-09-12T14:33:56.548 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:56.548 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-386 td/osd-backfill-space/datafile 2020-09-12T14:33:56.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:56.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-387 td/osd-backfill-space/datafile 2020-09-12T14:33:56.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:56.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-388 td/osd-backfill-space/datafile 2020-09-12T14:33:56.933 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:56.934 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-389 td/osd-backfill-space/datafile 2020-09-12T14:33:57.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:57.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-390 td/osd-backfill-space/datafile 2020-09-12T14:33:57.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:57.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-391 td/osd-backfill-space/datafile 2020-09-12T14:33:57.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:57.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-392 td/osd-backfill-space/datafile 2020-09-12T14:33:57.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:57.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-393 td/osd-backfill-space/datafile 2020-09-12T14:33:57.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:57.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-394 td/osd-backfill-space/datafile 2020-09-12T14:33:57.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:57.671 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-395 td/osd-backfill-space/datafile 2020-09-12T14:33:57.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:57.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-396 td/osd-backfill-space/datafile 2020-09-12T14:33:57.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:57.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-397 td/osd-backfill-space/datafile 2020-09-12T14:33:58.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:58.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-398 td/osd-backfill-space/datafile 2020-09-12T14:33:58.164 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:58.164 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-399 td/osd-backfill-space/datafile 2020-09-12T14:33:58.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:58.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-400 td/osd-backfill-space/datafile 2020-09-12T14:33:58.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:58.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-401 td/osd-backfill-space/datafile 2020-09-12T14:33:58.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:58.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-402 td/osd-backfill-space/datafile 2020-09-12T14:33:58.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:58.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-403 td/osd-backfill-space/datafile 2020-09-12T14:33:58.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:58.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-404 td/osd-backfill-space/datafile 2020-09-12T14:33:58.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:58.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-405 td/osd-backfill-space/datafile 2020-09-12T14:33:59.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:59.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-406 td/osd-backfill-space/datafile 2020-09-12T14:33:59.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:59.196 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-407 td/osd-backfill-space/datafile 2020-09-12T14:33:59.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:59.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-408 td/osd-backfill-space/datafile 2020-09-12T14:33:59.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:59.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-409 td/osd-backfill-space/datafile 2020-09-12T14:33:59.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:59.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-410 td/osd-backfill-space/datafile 2020-09-12T14:33:59.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:59.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-411 td/osd-backfill-space/datafile 2020-09-12T14:33:59.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:33:59.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-412 td/osd-backfill-space/datafile 2020-09-12T14:34:00.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:00.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-413 td/osd-backfill-space/datafile 2020-09-12T14:34:00.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:00.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-414 td/osd-backfill-space/datafile 2020-09-12T14:34:00.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:00.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-415 td/osd-backfill-space/datafile 2020-09-12T14:34:00.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:00.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-416 td/osd-backfill-space/datafile 2020-09-12T14:34:00.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:00.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-417 td/osd-backfill-space/datafile 2020-09-12T14:34:00.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:00.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-418 td/osd-backfill-space/datafile 2020-09-12T14:34:00.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:00.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-419 td/osd-backfill-space/datafile 2020-09-12T14:34:00.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:00.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-420 td/osd-backfill-space/datafile 2020-09-12T14:34:01.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:01.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-421 td/osd-backfill-space/datafile 2020-09-12T14:34:01.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:01.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-422 td/osd-backfill-space/datafile 2020-09-12T14:34:01.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:01.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-423 td/osd-backfill-space/datafile 2020-09-12T14:34:01.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:01.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-424 td/osd-backfill-space/datafile 2020-09-12T14:34:01.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:01.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-425 td/osd-backfill-space/datafile 2020-09-12T14:34:01.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:01.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-426 td/osd-backfill-space/datafile 2020-09-12T14:34:01.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:01.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-427 td/osd-backfill-space/datafile 2020-09-12T14:34:01.892 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:01.892 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-428 td/osd-backfill-space/datafile 2020-09-12T14:34:02.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:02.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-429 td/osd-backfill-space/datafile 2020-09-12T14:34:02.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:02.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-430 td/osd-backfill-space/datafile 2020-09-12T14:34:02.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:02.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-431 td/osd-backfill-space/datafile 2020-09-12T14:34:02.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:02.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-432 td/osd-backfill-space/datafile 2020-09-12T14:34:02.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:02.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-433 td/osd-backfill-space/datafile 2020-09-12T14:34:02.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:02.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-434 td/osd-backfill-space/datafile 2020-09-12T14:34:02.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:02.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-435 td/osd-backfill-space/datafile 2020-09-12T14:34:02.914 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:02.914 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-436 td/osd-backfill-space/datafile 2020-09-12T14:34:03.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:03.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-437 td/osd-backfill-space/datafile 2020-09-12T14:34:03.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:03.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-438 td/osd-backfill-space/datafile 2020-09-12T14:34:03.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:03.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-439 td/osd-backfill-space/datafile 2020-09-12T14:34:03.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:03.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-440 td/osd-backfill-space/datafile 2020-09-12T14:34:03.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:03.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-441 td/osd-backfill-space/datafile 2020-09-12T14:34:03.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:03.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-442 td/osd-backfill-space/datafile 2020-09-12T14:34:03.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:03.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-443 td/osd-backfill-space/datafile 2020-09-12T14:34:03.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:03.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-444 td/osd-backfill-space/datafile 2020-09-12T14:34:04.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:04.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-445 td/osd-backfill-space/datafile 2020-09-12T14:34:04.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:04.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-446 td/osd-backfill-space/datafile 2020-09-12T14:34:04.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:04.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-447 td/osd-backfill-space/datafile 2020-09-12T14:34:04.438 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:04.439 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-448 td/osd-backfill-space/datafile 2020-09-12T14:34:04.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:04.573 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-449 td/osd-backfill-space/datafile 2020-09-12T14:34:04.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:04.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-450 td/osd-backfill-space/datafile 2020-09-12T14:34:04.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:04.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-451 td/osd-backfill-space/datafile 2020-09-12T14:34:04.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:04.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-452 td/osd-backfill-space/datafile 2020-09-12T14:34:05.066 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:05.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-453 td/osd-backfill-space/datafile 2020-09-12T14:34:05.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:05.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-454 td/osd-backfill-space/datafile 2020-09-12T14:34:05.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:05.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-455 td/osd-backfill-space/datafile 2020-09-12T14:34:05.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:05.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-456 td/osd-backfill-space/datafile 2020-09-12T14:34:05.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:05.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-457 td/osd-backfill-space/datafile 2020-09-12T14:34:05.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:05.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-458 td/osd-backfill-space/datafile 2020-09-12T14:34:05.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:05.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-459 td/osd-backfill-space/datafile 2020-09-12T14:34:05.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:05.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-460 td/osd-backfill-space/datafile 2020-09-12T14:34:06.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:06.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-461 td/osd-backfill-space/datafile 2020-09-12T14:34:06.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:06.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-462 td/osd-backfill-space/datafile 2020-09-12T14:34:06.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:06.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-463 td/osd-backfill-space/datafile 2020-09-12T14:34:06.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:06.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-464 td/osd-backfill-space/datafile 2020-09-12T14:34:06.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:06.557 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-465 td/osd-backfill-space/datafile 2020-09-12T14:34:06.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:06.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-466 td/osd-backfill-space/datafile 2020-09-12T14:34:06.841 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:06.841 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-467 td/osd-backfill-space/datafile 2020-09-12T14:34:06.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:06.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-468 td/osd-backfill-space/datafile 2020-09-12T14:34:07.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:07.093 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-469 td/osd-backfill-space/datafile 2020-09-12T14:34:07.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:07.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-470 td/osd-backfill-space/datafile 2020-09-12T14:34:07.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:07.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-471 td/osd-backfill-space/datafile 2020-09-12T14:34:07.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:07.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-472 td/osd-backfill-space/datafile 2020-09-12T14:34:07.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:07.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-473 td/osd-backfill-space/datafile 2020-09-12T14:34:07.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:07.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-474 td/osd-backfill-space/datafile 2020-09-12T14:34:07.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:07.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-475 td/osd-backfill-space/datafile 2020-09-12T14:34:07.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:07.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-476 td/osd-backfill-space/datafile 2020-09-12T14:34:08.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:08.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-477 td/osd-backfill-space/datafile 2020-09-12T14:34:08.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:08.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-478 td/osd-backfill-space/datafile 2020-09-12T14:34:08.315 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:08.316 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-479 td/osd-backfill-space/datafile 2020-09-12T14:34:08.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:08.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-480 td/osd-backfill-space/datafile 2020-09-12T14:34:08.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:08.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-481 td/osd-backfill-space/datafile 2020-09-12T14:34:08.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:08.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-482 td/osd-backfill-space/datafile 2020-09-12T14:34:08.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:08.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-483 td/osd-backfill-space/datafile 2020-09-12T14:34:09.178 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:09.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-484 td/osd-backfill-space/datafile 2020-09-12T14:34:09.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:09.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-485 td/osd-backfill-space/datafile 2020-09-12T14:34:09.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:09.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-486 td/osd-backfill-space/datafile 2020-09-12T14:34:09.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:09.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-487 td/osd-backfill-space/datafile 2020-09-12T14:34:09.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:09.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-488 td/osd-backfill-space/datafile 2020-09-12T14:34:09.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:09.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-489 td/osd-backfill-space/datafile 2020-09-12T14:34:09.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:09.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-490 td/osd-backfill-space/datafile 2020-09-12T14:34:10.083 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:10.083 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-491 td/osd-backfill-space/datafile 2020-09-12T14:34:10.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:10.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-492 td/osd-backfill-space/datafile 2020-09-12T14:34:10.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:10.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-493 td/osd-backfill-space/datafile 2020-09-12T14:34:10.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:10.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-494 td/osd-backfill-space/datafile 2020-09-12T14:34:10.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:10.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-495 td/osd-backfill-space/datafile 2020-09-12T14:34:10.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:10.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-496 td/osd-backfill-space/datafile 2020-09-12T14:34:10.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:10.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-497 td/osd-backfill-space/datafile 2020-09-12T14:34:10.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:10.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-498 td/osd-backfill-space/datafile 2020-09-12T14:34:11.113 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:11.113 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-499 td/osd-backfill-space/datafile 2020-09-12T14:34:11.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:11.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-500 td/osd-backfill-space/datafile 2020-09-12T14:34:11.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:11.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-501 td/osd-backfill-space/datafile 2020-09-12T14:34:11.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:11.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-502 td/osd-backfill-space/datafile 2020-09-12T14:34:11.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:11.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-503 td/osd-backfill-space/datafile 2020-09-12T14:34:11.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:11.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-504 td/osd-backfill-space/datafile 2020-09-12T14:34:11.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:11.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-505 td/osd-backfill-space/datafile 2020-09-12T14:34:12.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:12.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-506 td/osd-backfill-space/datafile 2020-09-12T14:34:12.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:12.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-507 td/osd-backfill-space/datafile 2020-09-12T14:34:12.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:12.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-508 td/osd-backfill-space/datafile 2020-09-12T14:34:12.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:12.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-509 td/osd-backfill-space/datafile 2020-09-12T14:34:12.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:12.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-510 td/osd-backfill-space/datafile 2020-09-12T14:34:12.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:12.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-511 td/osd-backfill-space/datafile 2020-09-12T14:34:12.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:12.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-512 td/osd-backfill-space/datafile 2020-09-12T14:34:12.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:12.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-513 td/osd-backfill-space/datafile 2020-09-12T14:34:13.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:13.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-514 td/osd-backfill-space/datafile 2020-09-12T14:34:13.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:13.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-515 td/osd-backfill-space/datafile 2020-09-12T14:34:13.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:13.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-516 td/osd-backfill-space/datafile 2020-09-12T14:34:13.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:13.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-517 td/osd-backfill-space/datafile 2020-09-12T14:34:13.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:13.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-518 td/osd-backfill-space/datafile 2020-09-12T14:34:13.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:13.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-519 td/osd-backfill-space/datafile 2020-09-12T14:34:13.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:13.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-520 td/osd-backfill-space/datafile 2020-09-12T14:34:13.944 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:13.944 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-521 td/osd-backfill-space/datafile 2020-09-12T14:34:14.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:14.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-522 td/osd-backfill-space/datafile 2020-09-12T14:34:14.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:14.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-523 td/osd-backfill-space/datafile 2020-09-12T14:34:14.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:14.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-524 td/osd-backfill-space/datafile 2020-09-12T14:34:14.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:14.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-525 td/osd-backfill-space/datafile 2020-09-12T14:34:14.664 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:14.664 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-526 td/osd-backfill-space/datafile 2020-09-12T14:34:14.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:14.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-527 td/osd-backfill-space/datafile 2020-09-12T14:34:14.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:14.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-528 td/osd-backfill-space/datafile 2020-09-12T14:34:15.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:15.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-529 td/osd-backfill-space/datafile 2020-09-12T14:34:15.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:15.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-530 td/osd-backfill-space/datafile 2020-09-12T14:34:15.284 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:15.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-531 td/osd-backfill-space/datafile 2020-09-12T14:34:15.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:15.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-532 td/osd-backfill-space/datafile 2020-09-12T14:34:15.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:15.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-533 td/osd-backfill-space/datafile 2020-09-12T14:34:15.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:15.653 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-534 td/osd-backfill-space/datafile 2020-09-12T14:34:15.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:15.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-535 td/osd-backfill-space/datafile 2020-09-12T14:34:15.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:15.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-536 td/osd-backfill-space/datafile 2020-09-12T14:34:16.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:16.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-537 td/osd-backfill-space/datafile 2020-09-12T14:34:16.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:16.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-538 td/osd-backfill-space/datafile 2020-09-12T14:34:16.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:16.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-539 td/osd-backfill-space/datafile 2020-09-12T14:34:16.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:16.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-540 td/osd-backfill-space/datafile 2020-09-12T14:34:16.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:16.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-541 td/osd-backfill-space/datafile 2020-09-12T14:34:16.649 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:16.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-542 td/osd-backfill-space/datafile 2020-09-12T14:34:16.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:16.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-543 td/osd-backfill-space/datafile 2020-09-12T14:34:16.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:16.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-544 td/osd-backfill-space/datafile 2020-09-12T14:34:17.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:17.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-545 td/osd-backfill-space/datafile 2020-09-12T14:34:17.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:17.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-546 td/osd-backfill-space/datafile 2020-09-12T14:34:17.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:17.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-547 td/osd-backfill-space/datafile 2020-09-12T14:34:17.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:17.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-548 td/osd-backfill-space/datafile 2020-09-12T14:34:17.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:17.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-549 td/osd-backfill-space/datafile 2020-09-12T14:34:17.663 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:17.663 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-550 td/osd-backfill-space/datafile 2020-09-12T14:34:17.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:17.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-551 td/osd-backfill-space/datafile 2020-09-12T14:34:17.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:17.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-552 td/osd-backfill-space/datafile 2020-09-12T14:34:18.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:18.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-553 td/osd-backfill-space/datafile 2020-09-12T14:34:18.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:18.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-554 td/osd-backfill-space/datafile 2020-09-12T14:34:18.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:18.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-555 td/osd-backfill-space/datafile 2020-09-12T14:34:18.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:18.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-556 td/osd-backfill-space/datafile 2020-09-12T14:34:18.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:18.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-557 td/osd-backfill-space/datafile 2020-09-12T14:34:18.660 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:18.660 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-558 td/osd-backfill-space/datafile 2020-09-12T14:34:18.785 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:18.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-559 td/osd-backfill-space/datafile 2020-09-12T14:34:18.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:18.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-560 td/osd-backfill-space/datafile 2020-09-12T14:34:19.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:19.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-561 td/osd-backfill-space/datafile 2020-09-12T14:34:19.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:19.172 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-562 td/osd-backfill-space/datafile 2020-09-12T14:34:19.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:19.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-563 td/osd-backfill-space/datafile 2020-09-12T14:34:19.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:19.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-564 td/osd-backfill-space/datafile 2020-09-12T14:34:19.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:19.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-565 td/osd-backfill-space/datafile 2020-09-12T14:34:19.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:19.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-566 td/osd-backfill-space/datafile 2020-09-12T14:34:19.766 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:19.766 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-567 td/osd-backfill-space/datafile 2020-09-12T14:34:19.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:19.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-568 td/osd-backfill-space/datafile 2020-09-12T14:34:20.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:20.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-569 td/osd-backfill-space/datafile 2020-09-12T14:34:20.151 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:20.151 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-570 td/osd-backfill-space/datafile 2020-09-12T14:34:20.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:20.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-571 td/osd-backfill-space/datafile 2020-09-12T14:34:20.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:20.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-572 td/osd-backfill-space/datafile 2020-09-12T14:34:20.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:20.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-573 td/osd-backfill-space/datafile 2020-09-12T14:34:20.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:20.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-574 td/osd-backfill-space/datafile 2020-09-12T14:34:20.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:20.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-575 td/osd-backfill-space/datafile 2020-09-12T14:34:20.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:20.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-576 td/osd-backfill-space/datafile 2020-09-12T14:34:21.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:21.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-577 td/osd-backfill-space/datafile 2020-09-12T14:34:21.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:21.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-578 td/osd-backfill-space/datafile 2020-09-12T14:34:21.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:21.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-579 td/osd-backfill-space/datafile 2020-09-12T14:34:21.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:21.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-580 td/osd-backfill-space/datafile 2020-09-12T14:34:21.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:21.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-581 td/osd-backfill-space/datafile 2020-09-12T14:34:21.626 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:21.626 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-582 td/osd-backfill-space/datafile 2020-09-12T14:34:21.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:21.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-583 td/osd-backfill-space/datafile 2020-09-12T14:34:21.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:21.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-584 td/osd-backfill-space/datafile 2020-09-12T14:34:21.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:21.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-585 td/osd-backfill-space/datafile 2020-09-12T14:34:22.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:22.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-586 td/osd-backfill-space/datafile 2020-09-12T14:34:22.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:22.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-587 td/osd-backfill-space/datafile 2020-09-12T14:34:22.363 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:22.363 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-588 td/osd-backfill-space/datafile 2020-09-12T14:34:22.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:22.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-589 td/osd-backfill-space/datafile 2020-09-12T14:34:22.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:22.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-590 td/osd-backfill-space/datafile 2020-09-12T14:34:22.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:22.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-591 td/osd-backfill-space/datafile 2020-09-12T14:34:22.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:22.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-592 td/osd-backfill-space/datafile 2020-09-12T14:34:22.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:22.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-593 td/osd-backfill-space/datafile 2020-09-12T14:34:23.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:23.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-594 td/osd-backfill-space/datafile 2020-09-12T14:34:23.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:23.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-595 td/osd-backfill-space/datafile 2020-09-12T14:34:23.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:23.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-596 td/osd-backfill-space/datafile 2020-09-12T14:34:23.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:23.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-597 td/osd-backfill-space/datafile 2020-09-12T14:34:23.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:23.612 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-598 td/osd-backfill-space/datafile 2020-09-12T14:34:23.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:23.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-599 td/osd-backfill-space/datafile 2020-09-12T14:34:23.862 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:482: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:34:23.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:484: TEST_backfill_multi_partial: rados -p test1 put obj-1-600 td/osd-backfill-space/datafile 2020-09-12T14:34:23.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:487: TEST_backfill_multi_partial: ceph pg dump pgs 2020-09-12T14:34:24.361 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:34:24.361 INFO:tasks.workunit.client.0.smithi027.stdout:3.0 0 0 0 0 0 0 0 0 0 0 active+clean 2020-09-12T14:33:02.061981+0000 0'0 38:30 [2,0] 2 [2,0] 2 0'0 2020-09-12T14:32:03.946070+0000 0'0 2020-09-12T14:32:03.946070+0000 0 2020-09-12T14:34:24.361 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 542 0 0 0 0 1415162 0 0 42 42 stale+active+clean 2020-09-12T14:32:01.131349+0000 34'542 34:569 [1] 1 [1] 1 0'0 2020-09-12T14:31:58.829524+0000 0'0 2020-09-12T14:31:58.829524+0000 0 2020-09-12T14:34:24.361 INFO:tasks.workunit.client.0.smithi027.stdout:2.0 588 0 0 0 0 602112 0 0 88 88 active+clean 2020-09-12T14:33:02.062235+0000 39'588 39:630 [2,0] 2 [2,0] 2 0'0 2020-09-12T14:32:01.404494+0000 0'0 2020-09-12T14:32:01.404494+0000 0 2020-09-12T14:34:24.361 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:34:24.362 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:34:24.362 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:34:24.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:489: TEST_backfill_multi_partial: _objectstore_tool_nowait td/osd-backfill-space 2 --op export --pgid 2.0 --file td/osd-backfill-space/export.out 2020-09-12T14:34:24.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1207: _objectstore_tool_nowait: local dir=td/osd-backfill-space 2020-09-12T14:34:24.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1208: _objectstore_tool_nowait: shift 2020-09-12T14:34:24.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1209: _objectstore_tool_nowait: local id=2 2020-09-12T14:34:24.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1210: _objectstore_tool_nowait: shift 2020-09-12T14:34:24.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1212: _objectstore_tool_nowait: kill_daemons td/osd-backfill-space TERM osd.2 2020-09-12T14:34:24.373 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:34:24.373 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:34:24.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:34:24.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:34:24.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:34:24.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:34:24.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1214: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-backfill-space 2 --op export --pgid 2.0 --file td/osd-backfill-space/export.out 2020-09-12T14:34:24.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1195: _objectstore_tool_nodown: local dir=td/osd-backfill-space 2020-09-12T14:34:24.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: _objectstore_tool_nodown: shift 2020-09-12T14:34:24.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: _objectstore_tool_nodown: local id=2 2020-09-12T14:34:24.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1198: _objectstore_tool_nodown: shift 2020-09-12T14:34:24.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: _objectstore_tool_nodown: local osd_data=td/osd-backfill-space/2 2020-09-12T14:34:24.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1202: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-backfill-space/2 --op export --pgid 2.0 --file td/osd-backfill-space/export.out 2020-09-12T14:34:25.969 INFO:tasks.workunit.client.0.smithi027.stderr:Exporting 2.0 info 2.0( v 39'600 (39'500,39'600] local-lis/les=37/38 n=600 ec=26/26 lis/c=37/37 les/c/f=38/38/0 sis=37) 2020-09-12T14:34:25.976 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:00f101c6:::obj-1-121:head# 2020-09-12T14:34:25.977 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:0168051e:::obj-1-571:head# 2020-09-12T14:34:25.978 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:01b67738:::obj-1-182:head# 2020-09-12T14:34:25.978 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:040dbc59:::obj-1-23:head# 2020-09-12T14:34:25.985 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:0415ce6e:::obj-1-20:head# 2020-09-12T14:34:25.993 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:058b28d7:::obj-1-572:head# 2020-09-12T14:34:25.994 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:059d2e5c:::obj-1-87:head# 2020-09-12T14:34:25.995 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:05a823ca:::obj-1-52:head# 2020-09-12T14:34:25.995 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:05b8b7fe:::obj-1-119:head# 2020-09-12T14:34:25.995 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:05c9492b:::obj-1-78:head# 2020-09-12T14:34:25.996 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:05e91e96:::obj-1-393:head# 2020-09-12T14:34:25.996 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:07bdb780:::obj-1-204:head# 2020-09-12T14:34:25.997 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:07cfbd49:::obj-1-318:head# 2020-09-12T14:34:25.997 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:084a829f:::obj-1-112:head# 2020-09-12T14:34:25.998 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:08cedd34:::obj-1-438:head# 2020-09-12T14:34:25.998 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:0941af69:::obj-1-117:head# 2020-09-12T14:34:25.999 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:09607265:::obj-1-429:head# 2020-09-12T14:34:25.999 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:0be86f7e:::obj-1-568:head# 2020-09-12T14:34:26.000 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:0ca5819d:::obj-1-327:head# 2020-09-12T14:34:26.000 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:0cd02051:::obj-1-208:head# 2020-09-12T14:34:26.003 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:0d0264b5:::obj-1-171:head# 2020-09-12T14:34:26.003 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:0d195060:::obj-1-420:head# 2020-09-12T14:34:26.004 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:0d362b19:::obj-1-219:head# 2020-09-12T14:34:26.004 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:0dea6f4d:::obj-1-61:head# 2020-09-12T14:34:26.005 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:0ecd8390:::obj-1-132:head# 2020-09-12T14:34:26.005 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:0f49e2fa:::obj-1-200:head# 2020-09-12T14:34:26.006 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:0f7bb95f:::obj-1-268:head# 2020-09-12T14:34:26.014 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:0fabf252:::obj-1-441:head# 2020-09-12T14:34:26.015 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:0fb1697a:::obj-1-443:head# 2020-09-12T14:34:26.015 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:105d3efe:::obj-1-490:head# 2020-09-12T14:34:26.015 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:10f52c01:::obj-1-449:head# 2020-09-12T14:34:26.016 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:11965425:::obj-1-335:head# 2020-09-12T14:34:26.016 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:11c9f5f8:::obj-1-565:head# 2020-09-12T14:34:26.016 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:11cfcfb6:::obj-1-588:head# 2020-09-12T14:34:26.017 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:11d46f0a:::obj-1-387:head# 2020-09-12T14:34:26.017 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:11ebb0d6:::obj-1-214:head# 2020-09-12T14:34:26.017 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:1295e402:::obj-1-178:head# 2020-09-12T14:34:26.018 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:12c996c1:::obj-1-517:head# 2020-09-12T14:34:26.018 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:133dec15:::obj-1-475:head# 2020-09-12T14:34:26.019 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:1475c981:::obj-1-505:head# 2020-09-12T14:34:26.019 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:148e92a4:::obj-1-509:head# 2020-09-12T14:34:26.019 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:1490b121:::obj-1-172:head# 2020-09-12T14:34:26.019 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:14ce082c:::obj-1-99:head# 2020-09-12T14:34:26.020 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:1552ad99:::obj-1-41:head# 2020-09-12T14:34:26.020 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:15544270:::obj-1-271:head# 2020-09-12T14:34:26.021 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:1589d957:::obj-1-86:head# 2020-09-12T14:34:26.021 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:16db3947:::obj-1-477:head# 2020-09-12T14:34:26.021 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:16dc98bd:::obj-1-375:head# 2020-09-12T14:34:26.022 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:17e54db3:::obj-1-196:head# 2020-09-12T14:34:26.022 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:17f2ad3f:::obj-1-10:head# 2020-09-12T14:34:26.022 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:18075e41:::obj-1-55:head# 2020-09-12T14:34:26.023 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:18086253:::obj-1-71:head# 2020-09-12T14:34:26.023 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:191cf8d8:::obj-1-372:head# 2020-09-12T14:34:26.023 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:19311f70:::obj-1-454:head# 2020-09-12T14:34:26.024 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:198318e4:::obj-1-464:head# 2020-09-12T14:34:26.024 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:1999d806:::obj-1-163:head# 2020-09-12T14:34:26.025 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:1a92cab7:::obj-1-412:head# 2020-09-12T14:34:26.025 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:1b16c0f6:::obj-1-210:head# 2020-09-12T14:34:26.025 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:1b53ed46:::obj-1-577:head# 2020-09-12T14:34:26.026 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:1b5c5a8d:::obj-1-556:head# 2020-09-12T14:34:26.027 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:1c12c05e:::obj-1-79:head# 2020-09-12T14:34:26.027 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:1c571ba5:::obj-1-262:head# 2020-09-12T14:34:26.028 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:1c5f42b7:::obj-1-92:head# 2020-09-12T14:34:26.028 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:1c6ad881:::obj-1-312:head# 2020-09-12T14:34:26.029 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:1c89877b:::obj-1-142:head# 2020-09-12T14:34:26.029 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:1e4168ca:::obj-1-282:head# 2020-09-12T14:34:26.029 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:1e424445:::obj-1-484:head# 2020-09-12T14:34:26.029 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:1e467959:::obj-1-22:head# 2020-09-12T14:34:26.043 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:1eaf7e7c:::obj-1-392:head# 2020-09-12T14:34:26.044 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:1f06c86d:::obj-1-307:head# 2020-09-12T14:34:26.044 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:1f2ee1f8:::obj-1-306:head# 2020-09-12T14:34:26.045 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:1f4ef040:::obj-1-349:head# 2020-09-12T14:34:26.045 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:2039481f:::obj-1-545:head# 2020-09-12T14:34:26.045 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:22233df0:::obj-1-543:head# 2020-09-12T14:34:26.046 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:2225184e:::obj-1-266:head# 2020-09-12T14:34:26.046 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:234766bb:::obj-1-281:head# 2020-09-12T14:34:26.047 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:2372e022:::obj-1-205:head# 2020-09-12T14:34:26.047 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:2379547b:::obj-1-113:head# 2020-09-12T14:34:26.048 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:23ae14bf:::obj-1-531:head# 2020-09-12T14:34:26.048 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:23bc5f90:::obj-1-452:head# 2020-09-12T14:34:26.049 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:23fbef3d:::obj-1-82:head# 2020-09-12T14:34:26.049 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:24106c2b:::obj-1-232:head# 2020-09-12T14:34:26.050 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:2529b4e4:::obj-1-75:head# 2020-09-12T14:34:26.050 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:25a24ae7:::obj-1-459:head# 2020-09-12T14:34:26.051 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:26627aa9:::obj-1-7:head# 2020-09-12T14:34:26.051 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:26b77221:::obj-1-227:head# 2020-09-12T14:34:26.052 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:27450826:::obj-1-344:head# 2020-09-12T14:34:26.053 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:278905af:::obj-1-592:head# 2020-09-12T14:34:26.053 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:27ad0d2d:::obj-1-136:head# 2020-09-12T14:34:26.054 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:28a8c213:::obj-1-118:head# 2020-09-12T14:34:26.054 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:28c0c009:::obj-1-74:head# 2020-09-12T14:34:26.055 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:2a214708:::obj-1-260:head# 2020-09-12T14:34:26.055 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:2a67e195:::obj-1-474:head# 2020-09-12T14:34:26.055 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:2aa852ff:::obj-1-5:head# 2020-09-12T14:34:26.056 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:2ba1fb52:::obj-1-563:head# 2020-09-12T14:34:26.057 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:2bae8cb9:::obj-1-320:head# 2020-09-12T14:34:26.057 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:2c569d02:::obj-1-139:head# 2020-09-12T14:34:26.058 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:2c71d691:::obj-1-201:head# 2020-09-12T14:34:26.058 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:2c88e92c:::obj-1-389:head# 2020-09-12T14:34:26.058 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:2d659109:::obj-1-580:head# 2020-09-12T14:34:26.059 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:2d74ea60:::obj-1-230:head# 2020-09-12T14:34:26.062 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:2de96edf:::obj-1-233:head# 2020-09-12T14:34:26.063 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:2e43da40:::obj-1-382:head# 2020-09-12T14:34:26.063 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:2e530c9f:::obj-1-358:head# 2020-09-12T14:34:26.063 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:2ed02239:::obj-1-81:head# 2020-09-12T14:34:26.064 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:30264a0c:::obj-1-373:head# 2020-09-12T14:34:26.064 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:302ca5e7:::obj-1-206:head# 2020-09-12T14:34:26.065 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:309878e8:::obj-1-508:head# 2020-09-12T14:34:26.065 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:309a0ac3:::obj-1-188:head# 2020-09-12T14:34:26.065 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:30f030e9:::obj-1-53:head# 2020-09-12T14:34:26.066 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:31242a73:::obj-1-528:head# 2020-09-12T14:34:26.066 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:31463825:::obj-1-34:head# 2020-09-12T14:34:26.067 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3166dc86:::obj-1-355:head# 2020-09-12T14:34:26.067 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:31baa2fa:::obj-1-252:head# 2020-09-12T14:34:26.067 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:31d8935e:::obj-1-193:head# 2020-09-12T14:34:26.068 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:32670b14:::obj-1-305:head# 2020-09-12T14:34:26.068 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:33da02fa:::obj-1-114:head# 2020-09-12T14:34:26.068 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:34373133:::obj-1-446:head# 2020-09-12T14:34:26.069 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:347b7083:::obj-1-89:head# 2020-09-12T14:34:26.069 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:349125c1:::obj-1-148:head# 2020-09-12T14:34:26.070 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:34b80eee:::obj-1-138:head# 2020-09-12T14:34:26.070 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:34db48f8:::obj-1-49:head# 2020-09-12T14:34:26.071 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3669ff0c:::obj-1-313:head# 2020-09-12T14:34:26.071 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:36c1b610:::obj-1-249:head# 2020-09-12T14:34:26.071 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:374c250b:::obj-1-324:head# 2020-09-12T14:34:26.072 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3761a149:::obj-1-342:head# 2020-09-12T14:34:26.072 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:379ff95a:::obj-1-341:head# 2020-09-12T14:34:26.072 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:37dc15a6:::obj-1-26:head# 2020-09-12T14:34:26.073 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:38130e33:::obj-1-552:head# 2020-09-12T14:34:26.073 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:39c44880:::obj-1-538:head# 2020-09-12T14:34:26.073 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3a08dec9:::obj-1-368:head# 2020-09-12T14:34:26.074 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3a2b46a6:::obj-1-407:head# 2020-09-12T14:34:26.074 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3a7dfe0a:::obj-1-285:head# 2020-09-12T14:34:26.074 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3aebb144:::obj-1-248:head# 2020-09-12T14:34:26.075 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3b08b36f:::obj-1-310:head# 2020-09-12T14:34:26.075 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3b28ffa3:::obj-1-295:head# 2020-09-12T14:34:26.081 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3ba79bd1:::obj-1-444:head# 2020-09-12T14:34:26.082 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3c05df8d:::obj-1-319:head# 2020-09-12T14:34:26.083 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3c57e6a9:::obj-1-436:head# 2020-09-12T14:34:26.084 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3c73d8ee:::obj-1-298:head# 2020-09-12T14:34:26.084 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3cb6ae2a:::obj-1-400:head# 2020-09-12T14:34:26.085 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3cb71491:::obj-1-328:head# 2020-09-12T14:34:26.085 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3cb842fe:::obj-1-541:head# 2020-09-12T14:34:26.086 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3cf6835e:::obj-1-496:head# 2020-09-12T14:34:26.086 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3cf78507:::obj-1-573:head# 2020-09-12T14:34:26.087 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3d0dab84:::obj-1-554:head# 2020-09-12T14:34:26.087 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3d1a5f9d:::obj-1-323:head# 2020-09-12T14:34:26.087 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3da5f3a2:::obj-1-414:head# 2020-09-12T14:34:26.088 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3dbb65f0:::obj-1-440:head# 2020-09-12T14:34:26.089 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3dc94618:::obj-1-223:head# 2020-09-12T14:34:26.089 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3df3d5ab:::obj-1-25:head# 2020-09-12T14:34:26.090 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3e167988:::obj-1-336:head# 2020-09-12T14:34:26.091 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3e4ef3bd:::obj-1-104:head# 2020-09-12T14:34:26.091 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3e9e1efd:::obj-1-357:head# 2020-09-12T14:34:26.092 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3ebbe3a7:::obj-1-42:head# 2020-09-12T14:34:26.092 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3ec80373:::obj-1-403:head# 2020-09-12T14:34:26.093 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3ed086e4:::obj-1-13:head# 2020-09-12T14:34:26.093 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:3fe8e157:::obj-1-164:head# 2020-09-12T14:34:26.094 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:401ddfd4:::obj-1-379:head# 2020-09-12T14:34:26.094 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:4092f6cd:::obj-1-430:head# 2020-09-12T14:34:26.095 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:414bd7cc:::obj-1-562:head# 2020-09-12T14:34:26.095 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:422bd9d0:::obj-1-287:head# 2020-09-12T14:34:26.098 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:4255002d:::obj-1-578:head# 2020-09-12T14:34:26.099 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:4270725c:::obj-1-109:head# 2020-09-12T14:34:26.099 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:42836669:::obj-1-31:head# 2020-09-12T14:34:26.100 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:428d93ef:::obj-1-32:head# 2020-09-12T14:34:26.100 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:4327ef5b:::obj-1-453:head# 2020-09-12T14:34:26.101 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:440eb07b:::obj-1-236:head# 2020-09-12T14:34:26.101 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:442a028a:::obj-1-365:head# 2020-09-12T14:34:26.102 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:449ca54a:::obj-1-161:head# 2020-09-12T14:34:26.102 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:45128c3b:::obj-1-343:head# 2020-09-12T14:34:26.103 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:45801bf6:::obj-1-264:head# 2020-09-12T14:34:26.104 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:45896f61:::obj-1-48:head# 2020-09-12T14:34:26.104 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:459b7f1c:::obj-1-190:head# 2020-09-12T14:34:26.105 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:466af96a:::obj-1-486:head# 2020-09-12T14:34:26.105 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:47387ce4:::obj-1-575:head# 2020-09-12T14:34:26.106 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:47c94cc8:::obj-1-564:head# 2020-09-12T14:34:26.106 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:47d7f8ed:::obj-1-544:head# 2020-09-12T14:34:26.107 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:47fc0ec3:::obj-1-558:head# 2020-09-12T14:34:26.108 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:48ba42d8:::obj-1-156:head# 2020-09-12T14:34:26.108 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:494e269e:::obj-1-600:head# 2020-09-12T14:34:26.109 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:49646095:::obj-1-73:head# 2020-09-12T14:34:26.109 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:49eb65ab:::obj-1-374:head# 2020-09-12T14:34:26.109 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:4a0075a4:::obj-1-432:head# 2020-09-12T14:34:26.110 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:4af1ffc2:::obj-1-72:head# 2020-09-12T14:34:26.110 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:4bc1c46f:::obj-1-346:head# 2020-09-12T14:34:26.111 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:4bd55e68:::obj-1-93:head# 2020-09-12T14:34:26.111 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:4c6d2760:::obj-1-198:head# 2020-09-12T14:34:26.112 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:4cb9be3b:::obj-1-451:head# 2020-09-12T14:34:26.112 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:4d260a64:::obj-1-514:head# 2020-09-12T14:34:26.117 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:4d7d6284:::obj-1-583:head# 2020-09-12T14:34:26.118 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:4e79ac02:::obj-1-437:head# 2020-09-12T14:34:26.118 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:4e980099:::obj-1-507:head# 2020-09-12T14:34:26.119 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:4ea8f99c:::obj-1-211:head# 2020-09-12T14:34:26.119 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:4eabdc2a:::obj-1-590:head# 2020-09-12T14:34:26.121 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:4f97fd73:::obj-1-243:head# 2020-09-12T14:34:26.121 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:4fe00224:::obj-1-229:head# 2020-09-12T14:34:26.122 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:4fead9ab:::obj-1-476:head# 2020-09-12T14:34:26.123 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:4ffddfd7:::obj-1-462:head# 2020-09-12T14:34:26.124 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:5085c04d:::obj-1-100:head# 2020-09-12T14:34:26.124 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:50a8b465:::obj-1-574:head# 2020-09-12T14:34:26.125 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:518a151f:::obj-1-189:head# 2020-09-12T14:34:26.125 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:519afc95:::obj-1-314:head# 2020-09-12T14:34:26.126 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:52199781:::obj-1-202:head# 2020-09-12T14:34:26.126 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:52313062:::obj-1-337:head# 2020-09-12T14:34:26.127 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:526a5196:::obj-1-225:head# 2020-09-12T14:34:26.127 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:53128d32:::obj-1-483:head# 2020-09-12T14:34:26.127 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:53c2bb21:::obj-1-532:head# 2020-09-12T14:34:26.128 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:53fa4f3c:::obj-1-423:head# 2020-09-12T14:34:26.128 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:541c1d30:::obj-1-143:head# 2020-09-12T14:34:26.129 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:548d20c3:::obj-1-352:head# 2020-09-12T14:34:26.129 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:54e4c6f9:::obj-1-101:head# 2020-09-12T14:34:26.130 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:55016e06:::obj-1-276:head# 2020-09-12T14:34:26.131 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:5510c784:::obj-1-491:head# 2020-09-12T14:34:26.131 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:551275ef:::obj-1-239:head# 2020-09-12T14:34:26.132 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:556df001:::obj-1-46:head# 2020-09-12T14:34:26.132 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:558fb5b1:::obj-1-536:head# 2020-09-12T14:34:26.133 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:56390d98:::obj-1-64:head# 2020-09-12T14:34:26.133 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:564f962a:::obj-1-125:head# 2020-09-12T14:34:26.134 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:56936e94:::obj-1-251:head# 2020-09-12T14:34:26.134 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:57585d3d:::obj-1-294:head# 2020-09-12T14:34:26.134 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:5764cf7b:::obj-1-479:head# 2020-09-12T14:34:26.135 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:5782341a:::obj-1-218:head# 2020-09-12T14:34:26.136 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:57d07a1a:::obj-1-317:head# 2020-09-12T14:34:26.136 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:58700205:::obj-1-176:head# 2020-09-12T14:34:26.137 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:58aee718:::obj-1-434:head# 2020-09-12T14:34:26.137 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:594c9d52:::obj-1-458:head# 2020-09-12T14:34:26.138 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:5a70b555:::obj-1-279:head# 2020-09-12T14:34:26.139 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:5b02568b:::obj-1-586:head# 2020-09-12T14:34:26.139 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:5b69cdcf:::obj-1-391:head# 2020-09-12T14:34:26.140 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:5cf151b5:::obj-1-510:head# 2020-09-12T14:34:26.141 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:5d7fd07f:::obj-1-152:head# 2020-09-12T14:34:26.141 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:5de288d7:::obj-1-235:head# 2020-09-12T14:34:26.142 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:5ec497cd:::obj-1-44:head# 2020-09-12T14:34:26.142 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:5fb3a09f:::obj-1-518:head# 2020-09-12T14:34:26.143 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:5ff3c2ef:::obj-1-470:head# 2020-09-12T14:34:26.143 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:601efcb1:::obj-1-549:head# 2020-09-12T14:34:26.143 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:60831338:::obj-1-263:head# 2020-09-12T14:34:26.144 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:6092777b:::obj-1-145:head# 2020-09-12T14:34:26.144 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:60b17206:::obj-1-411:head# 2020-09-12T14:34:26.145 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:610c957c:::obj-1-278:head# 2020-09-12T14:34:26.146 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:613b5ef4:::obj-1-504:head# 2020-09-12T14:34:26.147 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:61ed7d60:::obj-1-274:head# 2020-09-12T14:34:26.148 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:621b6751:::obj-1-212:head# 2020-09-12T14:34:26.148 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:62b0aba9:::obj-1-253:head# 2020-09-12T14:34:26.149 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:64017240:::obj-1-102:head# 2020-09-12T14:34:26.149 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:645f4f49:::obj-1-332:head# 2020-09-12T14:34:26.150 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:65db82b9:::obj-1-334:head# 2020-09-12T14:34:26.150 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:667422e5:::obj-1-465:head# 2020-09-12T14:34:26.151 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:66c50bc2:::obj-1-322:head# 2020-09-12T14:34:26.151 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:681a4577:::obj-1-511:head# 2020-09-12T14:34:26.152 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:6874e007:::obj-1-398:head# 2020-09-12T14:34:26.152 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:693746c1:::obj-1-582:head# 2020-09-12T14:34:26.153 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:69638b0c:::obj-1-8:head# 2020-09-12T14:34:26.154 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:6967de6e:::obj-1-331:head# 2020-09-12T14:34:26.154 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:69da97aa:::obj-1-340:head# 2020-09-12T14:34:26.155 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:6b3a50a6:::obj-1-144:head# 2020-09-12T14:34:26.155 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:6be4ef26:::obj-1-579:head# 2020-09-12T14:34:26.155 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:6d735549:::obj-1-599:head# 2020-09-12T14:34:26.156 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:6e22c25e:::obj-1-9:head# 2020-09-12T14:34:26.156 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:6e2ced4e:::obj-1-370:head# 2020-09-12T14:34:26.157 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:6e961169:::obj-1-428:head# 2020-09-12T14:34:26.157 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:6efcbbee:::obj-1-168:head# 2020-09-12T14:34:26.158 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:6f20741c:::obj-1-33:head# 2020-09-12T14:34:26.158 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:6fe4c885:::obj-1-273:head# 2020-09-12T14:34:26.158 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:6fffdea2:::obj-1-425:head# 2020-09-12T14:34:26.159 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:70716852:::obj-1-593:head# 2020-09-12T14:34:26.160 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:70919606:::obj-1-515:head# 2020-09-12T14:34:26.160 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:70c39d76:::obj-1-467:head# 2020-09-12T14:34:26.161 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:70f57ad3:::obj-1-361:head# 2020-09-12T14:34:26.161 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:7177c7b5:::obj-1-503:head# 2020-09-12T14:34:26.161 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:719b69bd:::obj-1-497:head# 2020-09-12T14:34:26.162 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:724f9347:::obj-1-6:head# 2020-09-12T14:34:26.163 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:726d9f6b:::obj-1-111:head# 2020-09-12T14:34:26.163 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:72a91baf:::obj-1-439:head# 2020-09-12T14:34:26.164 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:72aa43ee:::obj-1-534:head# 2020-09-12T14:34:26.164 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:72c40e11:::obj-1-288:head# 2020-09-12T14:34:26.165 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:74196d45:::obj-1-246:head# 2020-09-12T14:34:26.165 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:743f42b1:::obj-1-326:head# 2020-09-12T14:34:26.166 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:7523b538:::obj-1-494:head# 2020-09-12T14:34:26.166 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:75310ed6:::obj-1-364:head# 2020-09-12T14:34:26.167 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:7568e0bc:::obj-1-585:head# 2020-09-12T14:34:26.167 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:760219a3:::obj-1-353:head# 2020-09-12T14:34:26.168 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:763338c9:::obj-1-555:head# 2020-09-12T14:34:26.168 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:77bc2d65:::obj-1-386:head# 2020-09-12T14:34:26.169 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:77ed0c7a:::obj-1-492:head# 2020-09-12T14:34:26.169 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:785a4052:::obj-1-456:head# 2020-09-12T14:34:26.170 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:796799c3:::obj-1-195:head# 2020-09-12T14:34:26.170 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:79caf0c1:::obj-1-445:head# 2020-09-12T14:34:26.171 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:7af5b3e4:::obj-1-217:head# 2020-09-12T14:34:26.172 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:7b3a9063:::obj-1-169:head# 2020-09-12T14:34:26.172 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:7b509a92:::obj-1-561:head# 2020-09-12T14:34:26.173 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:7bc646b1:::obj-1-415:head# 2020-09-12T14:34:26.174 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:7c2f9188:::obj-1-482:head# 2020-09-12T14:34:26.174 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:7d0da936:::obj-1-11:head# 2020-09-12T14:34:26.174 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:7d25ca2d:::obj-1-242:head# 2020-09-12T14:34:26.175 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:7d64f81e:::obj-1-473:head# 2020-09-12T14:34:26.175 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:7ef2ea1b:::obj-1-417:head# 2020-09-12T14:34:26.176 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:7f52b238:::obj-1-308:head# 2020-09-12T14:34:26.180 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:7f7a313c:::obj-1-551:head# 2020-09-12T14:34:26.182 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:806827a7:::obj-1-472:head# 2020-09-12T14:34:26.185 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:8112c3ab:::obj-1-45:head# 2020-09-12T14:34:26.185 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:81365b55:::obj-1-315:head# 2020-09-12T14:34:26.186 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:818ffdc3:::obj-1-345:head# 2020-09-12T14:34:26.186 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:81babab8:::obj-1-535:head# 2020-09-12T14:34:26.186 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:8288f652:::obj-1-347:head# 2020-09-12T14:34:26.186 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:8312eadd:::obj-1-14:head# 2020-09-12T14:34:26.186 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:83b89bc7:::obj-1-137:head# 2020-09-12T14:34:26.187 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:84a12321:::obj-1-134:head# 2020-09-12T14:34:26.187 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:84b26d62:::obj-1-408:head# 2020-09-12T14:34:26.187 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:84bbb26e:::obj-1-43:head# 2020-09-12T14:34:26.188 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:86fc0014:::obj-1-222:head# 2020-09-12T14:34:26.188 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:87488bf4:::obj-1-174:head# 2020-09-12T14:34:26.188 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:8754389a:::obj-1-65:head# 2020-09-12T14:34:26.188 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:87a6ad67:::obj-1-366:head# 2020-09-12T14:34:26.189 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:87d32b83:::obj-1-513:head# 2020-09-12T14:34:26.189 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:8813a346:::obj-1-540:head# 2020-09-12T14:34:26.189 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:88146951:::obj-1-216:head# 2020-09-12T14:34:26.190 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:8820800d:::obj-1-83:head# 2020-09-12T14:34:26.190 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:88319745:::obj-1-98:head# 2020-09-12T14:34:26.191 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:88c969b8:::obj-1-54:head# 2020-09-12T14:34:26.191 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:895a2d53:::obj-1-150:head# 2020-09-12T14:34:26.191 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:8ae006a0:::obj-1-241:head# 2020-09-12T14:34:26.192 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:8bd44e64:::obj-1-300:head# 2020-09-12T14:34:26.192 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:8beb257f:::obj-1-3:head# 2020-09-12T14:34:26.193 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:8cd8c743:::obj-1-527:head# 2020-09-12T14:34:26.194 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:8daa99c7:::obj-1-237:head# 2020-09-12T14:34:26.194 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:8f5bccbf:::obj-1-59:head# 2020-09-12T14:34:26.195 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:8f6d9945:::obj-1-151:head# 2020-09-12T14:34:26.195 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:8f7c568c:::obj-1-261:head# 2020-09-12T14:34:26.196 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:8fbc59a9:::obj-1-589:head# 2020-09-12T14:34:26.196 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:90225d94:::obj-1-194:head# 2020-09-12T14:34:26.197 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:904c50d3:::obj-1-63:head# 2020-09-12T14:34:26.197 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:90e1bd79:::obj-1-450:head# 2020-09-12T14:34:26.198 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:9274b06d:::obj-1-460:head# 2020-09-12T14:34:26.198 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:92af41f5:::obj-1-409:head# 2020-09-12T14:34:26.199 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:935a08d6:::obj-1-192:head# 2020-09-12T14:34:26.199 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:935f8a13:::obj-1-24:head# 2020-09-12T14:34:26.200 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:95230148:::obj-1-280:head# 2020-09-12T14:34:26.200 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:95342eb0:::obj-1-390:head# 2020-09-12T14:34:26.201 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:9659d09c:::obj-1-292:head# 2020-09-12T14:34:26.202 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:970e8cb4:::obj-1-18:head# 2020-09-12T14:34:26.202 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:97d21cf1:::obj-1-80:head# 2020-09-12T14:34:26.202 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:982bdb12:::obj-1-302:head# 2020-09-12T14:34:26.203 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:985dfd75:::obj-1-91:head# 2020-09-12T14:34:26.203 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:986d245a:::obj-1-154:head# 2020-09-12T14:34:26.204 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:98a03e99:::obj-1-529:head# 2020-09-12T14:34:26.204 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:98b7103e:::obj-1-275:head# 2020-09-12T14:34:26.204 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:990f0ecf:::obj-1-427:head# 2020-09-12T14:34:26.205 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:996c1f26:::obj-1-96:head# 2020-09-12T14:34:26.205 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:99bd2018:::obj-1-17:head# 2020-09-12T14:34:26.206 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:9a7434c9:::obj-1-469:head# 2020-09-12T14:34:26.206 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:9a8e36cc:::obj-1-598:head# 2020-09-12T14:34:26.207 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:9abcc369:::obj-1-442:head# 2020-09-12T14:34:26.208 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:9b0ea06e:::obj-1-231:head# 2020-09-12T14:34:26.209 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:9b34e80d:::obj-1-378:head# 2020-09-12T14:34:26.209 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:9b55a2a9:::obj-1-60:head# 2020-09-12T14:34:26.210 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:9be3f2da:::obj-1-597:head# 2020-09-12T14:34:26.210 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:9bf62c88:::obj-1-447:head# 2020-09-12T14:34:26.211 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:9c31857b:::obj-1-272:head# 2020-09-12T14:34:26.211 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:9c533e91:::obj-1-19:head# 2020-09-12T14:34:26.212 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:9ca70688:::obj-1-570:head# 2020-09-12T14:34:26.212 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:9ccce337:::obj-1-147:head# 2020-09-12T14:34:26.212 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:9d4b2dfc:::obj-1-267:head# 2020-09-12T14:34:26.213 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:9d4df969:::obj-1-286:head# 2020-09-12T14:34:26.213 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:9d714651:::obj-1-495:head# 2020-09-12T14:34:26.214 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:9da65518:::obj-1-498:head# 2020-09-12T14:34:26.214 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:9e94f27b:::obj-1-209:head# 2020-09-12T14:34:26.215 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:9f39e910:::obj-1-354:head# 2020-09-12T14:34:26.215 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:9f8f9ef4:::obj-1-226:head# 2020-09-12T14:34:26.215 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:9fcd3f36:::obj-1-47:head# 2020-09-12T14:34:26.216 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a0326d7d:::obj-1-338:head# 2020-09-12T14:34:26.216 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a05e7475:::obj-1-62:head# 2020-09-12T14:34:26.217 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a0955052:::obj-1-533:head# 2020-09-12T14:34:26.217 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a0aa6848:::obj-1-38:head# 2020-09-12T14:34:26.218 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a0fdc285:::obj-1-133:head# 2020-09-12T14:34:26.218 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a10290b8:::obj-1-245:head# 2020-09-12T14:34:26.219 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a118a8a8:::obj-1-500:head# 2020-09-12T14:34:26.219 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a11a4079:::obj-1-199:head# 2020-09-12T14:34:26.219 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a1b3ec36:::obj-1-15:head# 2020-09-12T14:34:26.220 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a1f5079b:::obj-1-468:head# 2020-09-12T14:34:26.221 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a1ff04b5:::obj-1-167:head# 2020-09-12T14:34:26.221 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a21cc437:::obj-1-128:head# 2020-09-12T14:34:26.222 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a2966c77:::obj-1-140:head# 2020-09-12T14:34:26.222 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a321e537:::obj-1-184:head# 2020-09-12T14:34:26.223 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a33418d8:::obj-1-488:head# 2020-09-12T14:34:26.223 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a38b1c1a:::obj-1-584:head# 2020-09-12T14:34:26.223 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a3f83221:::obj-1-158:head# 2020-09-12T14:34:26.224 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a46e33eb:::obj-1-106:head# 2020-09-12T14:34:26.224 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a491460d:::obj-1-421:head# 2020-09-12T14:34:26.225 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a4dd813b:::obj-1-530:head# 2020-09-12T14:34:26.226 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a4e0d96d:::obj-1-351:head# 2020-09-12T14:34:26.226 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a52657b4:::obj-1-557:head# 2020-09-12T14:34:26.227 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a5b3bca6:::obj-1-277:head# 2020-09-12T14:34:26.228 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a5f5a9fb:::obj-1-170:head# 2020-09-12T14:34:26.228 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a6aeed57:::obj-1-141:head# 2020-09-12T14:34:26.229 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a6dda70f:::obj-1-471:head# 2020-09-12T14:34:26.229 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a71186c0:::obj-1-406:head# 2020-09-12T14:34:26.230 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a764ad8c:::obj-1-221:head# 2020-09-12T14:34:26.230 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a7b63473:::obj-1-131:head# 2020-09-12T14:34:26.231 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a88042fa:::obj-1-296:head# 2020-09-12T14:34:26.231 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:a8805fe0:::obj-1-213:head# 2020-09-12T14:34:26.232 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:aa4f12b1:::obj-1-270:head# 2020-09-12T14:34:26.232 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:ab27f701:::obj-1-399:head# 2020-09-12T14:34:26.233 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:ab8027c1:::obj-1-360:head# 2020-09-12T14:34:26.234 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:ac3ef8ab:::obj-1-371:head# 2020-09-12T14:34:26.234 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:ac4db76d:::obj-1-405:head# 2020-09-12T14:34:26.235 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:ad53cb29:::obj-1-256:head# 2020-09-12T14:34:26.236 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:ad704c16:::obj-1-485:head# 2020-09-12T14:34:26.236 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:adec4920:::obj-1-107:head# 2020-09-12T14:34:26.236 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:ae3a94b3:::obj-1-95:head# 2020-09-12T14:34:26.237 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:ae92616e:::obj-1-448:head# 2020-09-12T14:34:26.237 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:ae975168:::obj-1-426:head# 2020-09-12T14:34:26.238 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:aee8fd62:::obj-1-27:head# 2020-09-12T14:34:26.238 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:af38aa87:::obj-1-50:head# 2020-09-12T14:34:26.239 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:b0327eb3:::obj-1-380:head# 2020-09-12T14:34:26.239 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:b03cd166:::obj-1-203:head# 2020-09-12T14:34:26.240 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:b06f06e0:::obj-1-126:head# 2020-09-12T14:34:26.241 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:b0972538:::obj-1-506:head# 2020-09-12T14:34:26.241 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:b0cfb86a:::obj-1-359:head# 2020-09-12T14:34:26.242 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:b10adb1c:::obj-1-333:head# 2020-09-12T14:34:26.242 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:b20b9004:::obj-1-401:head# 2020-09-12T14:34:26.242 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:b2113757:::obj-1-4:head# 2020-09-12T14:34:26.243 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:b22441e9:::obj-1-29:head# 2020-09-12T14:34:26.244 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:b2472384:::obj-1-207:head# 2020-09-12T14:34:26.244 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:b2604227:::obj-1-502:head# 2020-09-12T14:34:26.245 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:b3277f89:::obj-1-175:head# 2020-09-12T14:34:26.245 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:b3d09e0d:::obj-1-258:head# 2020-09-12T14:34:26.246 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:b3e9969a:::obj-1-499:head# 2020-09-12T14:34:26.247 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:b4892e7d:::obj-1-155:head# 2020-09-12T14:34:26.247 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:b4b76084:::obj-1-581:head# 2020-09-12T14:34:26.247 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:b6e47ae5:::obj-1-215:head# 2020-09-12T14:34:26.248 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:b75af5ff:::obj-1-228:head# 2020-09-12T14:34:26.248 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:b8e2e20f:::obj-1-220:head# 2020-09-12T14:34:26.249 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:b8e80c4e:::obj-1-576:head# 2020-09-12T14:34:26.249 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:ba502aee:::obj-1-255:head# 2020-09-12T14:34:26.250 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:ba9c3e6e:::obj-1-57:head# 2020-09-12T14:34:26.250 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:bb068062:::obj-1-461:head# 2020-09-12T14:34:26.251 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:bb16c115:::obj-1-157:head# 2020-09-12T14:34:26.251 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:bb447914:::obj-1-481:head# 2020-09-12T14:34:26.251 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:bb6c8e0f:::obj-1-348:head# 2020-09-12T14:34:26.252 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:bb77d6d0:::obj-1-301:head# 2020-09-12T14:34:26.252 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:bc51c5f6:::obj-1-422:head# 2020-09-12T14:34:26.253 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:bc838d5c:::obj-1-35:head# 2020-09-12T14:34:26.254 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:bceae5eb:::obj-1-56:head# 2020-09-12T14:34:26.254 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:bea17ef6:::obj-1-84:head# 2020-09-12T14:34:26.254 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:beb88be5:::obj-1-396:head# 2020-09-12T14:34:26.255 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:beed7b18:::obj-1-569:head# 2020-09-12T14:34:26.255 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:bf0bff49:::obj-1-385:head# 2020-09-12T14:34:26.256 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:bf545d5a:::obj-1-69:head# 2020-09-12T14:34:26.257 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:bf560a7f:::obj-1-40:head# 2020-09-12T14:34:26.257 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:bff6999c:::obj-1-240:head# 2020-09-12T14:34:26.258 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:c0a33d38:::obj-1-321:head# 2020-09-12T14:34:26.258 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:c1e4c9b4:::obj-1-299:head# 2020-09-12T14:34:26.259 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:c1ec515a:::obj-1-16:head# 2020-09-12T14:34:26.259 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:c22de3c8:::obj-1-110:head# 2020-09-12T14:34:26.260 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:c231cd7c:::obj-1-58:head# 2020-09-12T14:34:26.260 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:c2ee6ace:::obj-1-186:head# 2020-09-12T14:34:26.261 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:c32bfef2:::obj-1-181:head# 2020-09-12T14:34:26.261 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:c575a298:::obj-1-466:head# 2020-09-12T14:34:26.262 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:c58c3f80:::obj-1-493:head# 2020-09-12T14:34:26.262 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:c6442cf2:::obj-1-455:head# 2020-09-12T14:34:26.263 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:c6619413:::obj-1-37:head# 2020-09-12T14:34:26.264 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:c6c2f8e5:::obj-1-519:head# 2020-09-12T14:34:26.264 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:c6f811a2:::obj-1-180:head# 2020-09-12T14:34:26.264 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:c72af975:::obj-1-21:head# 2020-09-12T14:34:26.268 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:c73eb350:::obj-1-356:head# 2020-09-12T14:34:26.269 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:c7a683da:::obj-1-410:head# 2020-09-12T14:34:26.269 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:c7eb9bcc:::obj-1-553:head# 2020-09-12T14:34:26.270 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:c886df99:::obj-1-269:head# 2020-09-12T14:34:26.271 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:c932e3ec:::obj-1-135:head# 2020-09-12T14:34:26.271 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:c9b6507a:::obj-1-520:head# 2020-09-12T14:34:26.272 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:c9e3b064:::obj-1-244:head# 2020-09-12T14:34:26.272 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:cb705d66:::obj-1-377:head# 2020-09-12T14:34:26.273 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:cc80e25c:::obj-1-115:head# 2020-09-12T14:34:26.273 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:cd1ec88e:::obj-1-297:head# 2020-09-12T14:34:26.274 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:cd7e6936:::obj-1-234:head# 2020-09-12T14:34:26.274 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:ce32b69c:::obj-1-30:head# 2020-09-12T14:34:26.275 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:cf2b123c:::obj-1-293:head# 2020-09-12T14:34:26.275 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:cf2e80a9:::obj-1-542:head# 2020-09-12T14:34:26.276 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:cf51da6a:::obj-1-284:head# 2020-09-12T14:34:26.276 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:cf590f42:::obj-1-381:head# 2020-09-12T14:34:26.277 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:cf68a2b1:::obj-1-68:head# 2020-09-12T14:34:26.277 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:cf6aefb6:::obj-1-28:head# 2020-09-12T14:34:26.278 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:cfd8e6af:::obj-1-105:head# 2020-09-12T14:34:26.278 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d0640bad:::obj-1-254:head# 2020-09-12T14:34:26.279 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d101cda5:::obj-1-259:head# 2020-09-12T14:34:26.279 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d1be9386:::obj-1-416:head# 2020-09-12T14:34:26.280 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d243f036:::obj-1-265:head# 2020-09-12T14:34:26.280 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d2808c34:::obj-1-159:head# 2020-09-12T14:34:26.283 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d3415824:::obj-1-39:head# 2020-09-12T14:34:26.284 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d424f07a:::obj-1-547:head# 2020-09-12T14:34:26.284 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d44139f2:::obj-1-289:head# 2020-09-12T14:34:26.285 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d46015ab:::obj-1-546:head# 2020-09-12T14:34:26.285 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d4e503db:::obj-1-525:head# 2020-09-12T14:34:26.286 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d4e97d18:::obj-1-85:head# 2020-09-12T14:34:26.288 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d511492e:::obj-1-388:head# 2020-09-12T14:34:26.288 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d526a7fa:::obj-1-12:head# 2020-09-12T14:34:26.289 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d5667b59:::obj-1-369:head# 2020-09-12T14:34:26.289 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d57f2261:::obj-1-419:head# 2020-09-12T14:34:26.289 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d57f281b:::obj-1-250:head# 2020-09-12T14:34:26.290 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d6e05894:::obj-1-166:head# 2020-09-12T14:34:26.291 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d7689481:::obj-1-424:head# 2020-09-12T14:34:26.291 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d7d27743:::obj-1-129:head# 2020-09-12T14:34:26.292 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d7e4d3e9:::obj-1-127:head# 2020-09-12T14:34:26.292 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d7f1422a:::obj-1-594:head# 2020-09-12T14:34:26.292 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d8469fd2:::obj-1-153:head# 2020-09-12T14:34:26.293 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d84efd9e:::obj-1-435:head# 2020-09-12T14:34:26.293 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d883004b:::obj-1-162:head# 2020-09-12T14:34:26.294 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d8b60854:::obj-1-257:head# 2020-09-12T14:34:26.297 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d91aaae6:::obj-1-1:head# 2020-09-12T14:34:26.298 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d98695ca:::obj-1-524:head# 2020-09-12T14:34:26.298 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:d9ae97c8:::obj-1-177:head# 2020-09-12T14:34:26.299 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:da1713bd:::obj-1-309:head# 2020-09-12T14:34:26.299 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:da37824c:::obj-1-2:head# 2020-09-12T14:34:26.299 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:dab221b0:::obj-1-526:head# 2020-09-12T14:34:26.300 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:db27d885:::obj-1-363:head# 2020-09-12T14:34:26.300 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:dc001d76:::obj-1-329:head# 2020-09-12T14:34:26.301 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:dc4f4119:::obj-1-122:head# 2020-09-12T14:34:26.301 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:dcc36eb7:::obj-1-283:head# 2020-09-12T14:34:26.301 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:df7da06c:::obj-1-124:head# 2020-09-12T14:34:26.302 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:df7fcacb:::obj-1-330:head# 2020-09-12T14:34:26.302 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e05cbe31:::obj-1-413:head# 2020-09-12T14:34:26.303 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e1060a54:::obj-1-384:head# 2020-09-12T14:34:26.303 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e15b839b:::obj-1-316:head# 2020-09-12T14:34:26.304 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e161b6e5:::obj-1-103:head# 2020-09-12T14:34:26.304 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e1d1d79c:::obj-1-512:head# 2020-09-12T14:34:26.305 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e224b482:::obj-1-501:head# 2020-09-12T14:34:26.305 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e22d6f76:::obj-1-70:head# 2020-09-12T14:34:26.305 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e2bd801c:::obj-1-523:head# 2020-09-12T14:34:26.306 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e3d9f395:::obj-1-397:head# 2020-09-12T14:34:26.306 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e474ea25:::obj-1-457:head# 2020-09-12T14:34:26.307 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e49e1ff3:::obj-1-130:head# 2020-09-12T14:34:26.307 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e56c67a0:::obj-1-191:head# 2020-09-12T14:34:26.308 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e576e921:::obj-1-376:head# 2020-09-12T14:34:26.308 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e619fd26:::obj-1-395:head# 2020-09-12T14:34:26.309 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e625bdc5:::obj-1-522:head# 2020-09-12T14:34:26.309 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e68a0fa6:::obj-1-394:head# 2020-09-12T14:34:26.310 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e6cf260f:::obj-1-77:head# 2020-09-12T14:34:26.310 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e6d6255a:::obj-1-108:head# 2020-09-12T14:34:26.311 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e7a35d2f:::obj-1-116:head# 2020-09-12T14:34:26.311 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e7b07de5:::obj-1-238:head# 2020-09-12T14:34:26.311 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e8037aba:::obj-1-404:head# 2020-09-12T14:34:26.312 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e8e8b34f:::obj-1-291:head# 2020-09-12T14:34:26.312 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:e987cd7a:::obj-1-123:head# 2020-09-12T14:34:26.313 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:ea534db1:::obj-1-311:head# 2020-09-12T14:34:26.313 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:eabcc188:::obj-1-325:head# 2020-09-12T14:34:26.313 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:eb086660:::obj-1-595:head# 2020-09-12T14:34:26.313 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:eb2235d3:::obj-1-596:head# 2020-09-12T14:34:26.314 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:eb35ae40:::obj-1-516:head# 2020-09-12T14:34:26.314 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:eb360a13:::obj-1-224:head# 2020-09-12T14:34:26.314 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:eb3a3ea1:::obj-1-120:head# 2020-09-12T14:34:26.327 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:ebca9f6e:::obj-1-290:head# 2020-09-12T14:34:26.335 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:ec81db71:::obj-1-478:head# 2020-09-12T14:34:26.336 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:edeafa4c:::obj-1-362:head# 2020-09-12T14:34:26.355 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:eeb5261b:::obj-1-539:head# 2020-09-12T14:34:26.360 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:eec00eee:::obj-1-567:head# 2020-09-12T14:34:26.360 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:ef075e10:::obj-1-463:head# 2020-09-12T14:34:26.361 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:ef8e37d9:::obj-1-94:head# 2020-09-12T14:34:26.361 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:efad7f67:::obj-1-197:head# 2020-09-12T14:34:26.362 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:efe09ea7:::obj-1-537:head# 2020-09-12T14:34:26.362 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f0445019:::obj-1-480:head# 2020-09-12T14:34:26.363 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f08a432c:::obj-1-489:head# 2020-09-12T14:34:26.363 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f0bd9006:::obj-1-487:head# 2020-09-12T14:34:26.363 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f11a115d:::obj-1-591:head# 2020-09-12T14:34:26.364 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f232e0af:::obj-1-146:head# 2020-09-12T14:34:26.364 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f258db54:::obj-1-521:head# 2020-09-12T14:34:26.365 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f29cc204:::obj-1-566:head# 2020-09-12T14:34:26.365 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f2ae2451:::obj-1-66:head# 2020-09-12T14:34:26.365 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f318810a:::obj-1-160:head# 2020-09-12T14:34:26.366 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f3c35a0d:::obj-1-67:head# 2020-09-12T14:34:26.366 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f3ce9c90:::obj-1-88:head# 2020-09-12T14:34:26.366 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f3e3e81f:::obj-1-304:head# 2020-09-12T14:34:26.367 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f3f65b0a:::obj-1-367:head# 2020-09-12T14:34:26.367 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f4ea014b:::obj-1-149:head# 2020-09-12T14:34:26.367 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f52d115b:::obj-1-383:head# 2020-09-12T14:34:26.367 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f590311e:::obj-1-76:head# 2020-09-12T14:34:26.368 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f5a37734:::obj-1-51:head# 2020-09-12T14:34:26.368 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f5e2e77c:::obj-1-179:head# 2020-09-12T14:34:26.369 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f5e381bc:::obj-1-173:head# 2020-09-12T14:34:26.369 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f6c0ae0b:::obj-1-247:head# 2020-09-12T14:34:26.370 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f6db8a82:::obj-1-548:head# 2020-09-12T14:34:26.370 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f6e2dc46:::obj-1-36:head# 2020-09-12T14:34:26.371 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f76ee569:::obj-1-587:head# 2020-09-12T14:34:26.371 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f773b9ed:::obj-1-550:head# 2020-09-12T14:34:26.372 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f7e1adad:::obj-1-350:head# 2020-09-12T14:34:26.372 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f7eebf7a:::obj-1-559:head# 2020-09-12T14:34:26.372 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f877f025:::obj-1-187:head# 2020-09-12T14:34:26.372 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f992f9e9:::obj-1-402:head# 2020-09-12T14:34:26.373 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:f9fe960a:::obj-1-418:head# 2020-09-12T14:34:26.373 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:fa9b882f:::obj-1-165:head# 2020-09-12T14:34:26.374 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:fbd6461c:::obj-1-339:head# 2020-09-12T14:34:26.374 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:fbeab28d:::obj-1-431:head# 2020-09-12T14:34:26.374 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:fdea4138:::obj-1-185:head# 2020-09-12T14:34:26.375 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:fdfa7c9b:::obj-1-183:head# 2020-09-12T14:34:26.376 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:fe1ba57e:::obj-1-90:head# 2020-09-12T14:34:26.376 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:feb7adea:::obj-1-560:head# 2020-09-12T14:34:26.376 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:ffab306e:::obj-1-97:head# 2020-09-12T14:34:26.377 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:ffdfd397:::obj-1-433:head# 2020-09-12T14:34:26.377 INFO:tasks.workunit.client.0.smithi027.stderr:Read #2:ffe8cf04:::obj-1-303:head# 2020-09-12T14:34:26.377 INFO:tasks.workunit.client.0.smithi027.stderr:Export successful 2020-09-12T14:34:26.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1215: _objectstore_tool_nowait: activate_osd td/osd-backfill-space 2 2020-09-12T14:34:26.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/osd-backfill-space 2020-09-12T14:34:26.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T14:34:26.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=2 2020-09-12T14:34:26.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T14:34:26.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/osd-backfill-space/2 2020-09-12T14:34:26.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:34:26.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:34:26.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:34:26.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:34:26.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/osd-backfill-space/2' 2020-09-12T14:34:26.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/2/journal' 2020-09-12T14:34:26.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T14:34:26.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T14:34:26.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:34:26.648 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T14:34:26.648 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:34:26.648 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:34:26.649 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:34:26.649 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:34:26.649 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:34:26.649 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:34:26.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:34:26.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:34:26.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:34:26.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:34:26.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:34:26.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:34:26.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:34:26.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T14:34:26.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T14:34:26.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/osd-backfill-space/2 2020-09-12T14:34:26.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.2 2020-09-12T14:34:26.652 INFO:tasks.workunit.client.0.smithi027.stderr:start osd.2 2020-09-12T14:34:26.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 2 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/2 --osd-journal=td/osd-backfill-space/2/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:34:26.652 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/osd-backfill-space/2/whoami 2020-09-12T14:34:26.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 2 = 2 ']' 2020-09-12T14:34:26.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T14:34:26.653 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T14:34:26.653 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T14:34:26.675 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:34:26.675+0000 7fa71ac8aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:34:26.677 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:34:26.677+0000 7fa71ac8aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:34:26.678 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:34:26.678+0000 7fa71ac8aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:34:27.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: wait_for_osd up 2 2020-09-12T14:34:27.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:34:27.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T14:34:27.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:34:27.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:34:27.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:34:27.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:34:27.064 INFO:tasks.workunit.client.0.smithi027.stderr:0 2020-09-12T14:34:27.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:34:27.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:34:27.202 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:34:27.201+0000 7fa71ac8aec0 -1 Falling back to public interface 2020-09-12T14:34:27.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:34:28.069 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:34:28.068+0000 7fa71ac8aec0 -1 osd.2 39 log_to_monitors {default=true} 2020-09-12T14:34:28.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:34:28.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:34:28.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:34:28.475 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:34:28.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:34:28.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:34:28.968 INFO:tasks.workunit.client.0.smithi027.stderr:osd.2 up in weight 1 up_from 42 up_thru 42 down_at 40 last_clean_interval [15,39) [v2:127.0.0.1:6818/207395,v1:127.0.0.1:6819/207395] [v2:127.0.0.1:6820/207395,v1:127.0.0.1:6821/207395] exists,up bddb3624-216d-4188-90f9-94cad5cb9bd8 2020-09-12T14:34:28.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:34:28.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:34:28.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:34:28.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:490: TEST_backfill_multi_partial: kill_daemon td/osd-backfill-space/osd.1.pid TERM 2020-09-12T14:34:28.970 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:255: kill_daemon: cat td/osd-backfill-space/osd.1.pid 2020-09-12T14:34:28.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:255: kill_daemon: local pid=194660 2020-09-12T14:34:28.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:256: kill_daemon: local send_signal=TERM 2020-09-12T14:34:28.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:257: kill_daemon: local 'delays=0.1 0.2 1 1 1 2 3 5 5 5 10 10 20 60 60 60 120' 2020-09-12T14:34:28.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:258: kill_daemon: local exit_code=1 2020-09-12T14:34:28.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:261: kill_daemon: for try in $delays 0 2020-09-12T14:34:28.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: kill -TERM 194660 2020-09-12T14:34:28.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:263: kill_daemon: exit_code=1 2020-09-12T14:34:28.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:268: kill_daemon: send_signal=0 2020-09-12T14:34:28.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:269: kill_daemon: sleep 0.1 2020-09-12T14:34:29.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:261: kill_daemon: for try in $delays 0 2020-09-12T14:34:29.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: kill -0 194660 2020-09-12T14:34:29.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:265: kill_daemon: exit_code=0 2020-09-12T14:34:29.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:266: kill_daemon: break 2020-09-12T14:34:29.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:271: kill_daemon: return 0 2020-09-12T14:34:29.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:491: TEST_backfill_multi_partial: _objectstore_tool_nodown td/osd-backfill-space 1 --force --op remove --pgid 2.0 2020-09-12T14:34:29.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1195: _objectstore_tool_nodown: local dir=td/osd-backfill-space 2020-09-12T14:34:29.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: _objectstore_tool_nodown: shift 2020-09-12T14:34:29.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: _objectstore_tool_nodown: local id=1 2020-09-12T14:34:29.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1198: _objectstore_tool_nodown: shift 2020-09-12T14:34:29.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: _objectstore_tool_nodown: local osd_data=td/osd-backfill-space/1 2020-09-12T14:34:29.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1202: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-backfill-space/1 --force --op remove --pgid 2.0 2020-09-12T14:34:30.081 INFO:tasks.workunit.client.0.smithi027.stderr:PG '2.0' not found 2020-09-12T14:34:30.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1203: _objectstore_tool_nodown: return 1 2020-09-12T14:34:30.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:492: TEST_backfill_multi_partial: _objectstore_tool_nodown td/osd-backfill-space 1 --op import --pgid 2.0 --file td/osd-backfill-space/export.out 2020-09-12T14:34:30.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1195: _objectstore_tool_nodown: local dir=td/osd-backfill-space 2020-09-12T14:34:30.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: _objectstore_tool_nodown: shift 2020-09-12T14:34:30.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: _objectstore_tool_nodown: local id=1 2020-09-12T14:34:30.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1198: _objectstore_tool_nodown: shift 2020-09-12T14:34:30.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: _objectstore_tool_nodown: local osd_data=td/osd-backfill-space/1 2020-09-12T14:34:30.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1202: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-backfill-space/1 --op import --pgid 2.0 --file td/osd-backfill-space/export.out 2020-09-12T14:34:30.924 INFO:tasks.workunit.client.0.smithi027.stdout:get_pg_num_history pg_num_history pg_num_history(e43 pg_nums {1={21=1},2={26=1},3={31=1}} deleted_pools ) 2020-09-12T14:34:30.924 INFO:tasks.workunit.client.0.smithi027.stdout:Importing pgid 2.0 2020-09-12T14:34:30.924 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:00f101c6:::obj-1-121:head# 2020-09-12T14:34:30.925 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:30.974 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:0168051e:::obj-1-571:head# 2020-09-12T14:34:30.974 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:30.999 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:01b67738:::obj-1-182:head# 2020-09-12T14:34:30.999 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.024 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:040dbc59:::obj-1-23:head# 2020-09-12T14:34:31.025 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.066 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:0415ce6e:::obj-1-20:head# 2020-09-12T14:34:31.066 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.091 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:058b28d7:::obj-1-572:head# 2020-09-12T14:34:31.092 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.116 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:059d2e5c:::obj-1-87:head# 2020-09-12T14:34:31.117 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.141 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:05a823ca:::obj-1-52:head# 2020-09-12T14:34:31.142 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.166 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:05b8b7fe:::obj-1-119:head# 2020-09-12T14:34:31.167 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.192 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:05c9492b:::obj-1-78:head# 2020-09-12T14:34:31.192 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.217 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:05e91e96:::obj-1-393:head# 2020-09-12T14:34:31.217 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.242 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:07bdb780:::obj-1-204:head# 2020-09-12T14:34:31.242 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.267 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:07cfbd49:::obj-1-318:head# 2020-09-12T14:34:31.267 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.292 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:084a829f:::obj-1-112:head# 2020-09-12T14:34:31.293 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.326 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:08cedd34:::obj-1-438:head# 2020-09-12T14:34:31.326 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.351 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:0941af69:::obj-1-117:head# 2020-09-12T14:34:31.351 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.376 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:09607265:::obj-1-429:head# 2020-09-12T14:34:31.376 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.410 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:0be86f7e:::obj-1-568:head# 2020-09-12T14:34:31.410 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.443 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:0ca5819d:::obj-1-327:head# 2020-09-12T14:34:31.443 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.476 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:0cd02051:::obj-1-208:head# 2020-09-12T14:34:31.477 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.510 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:0d0264b5:::obj-1-171:head# 2020-09-12T14:34:31.510 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.543 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:0d195060:::obj-1-420:head# 2020-09-12T14:34:31.543 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.577 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:0d362b19:::obj-1-219:head# 2020-09-12T14:34:31.577 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.610 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:0dea6f4d:::obj-1-61:head# 2020-09-12T14:34:31.611 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.644 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:0ecd8390:::obj-1-132:head# 2020-09-12T14:34:31.645 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.677 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:0f49e2fa:::obj-1-200:head# 2020-09-12T14:34:31.678 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.711 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:0f7bb95f:::obj-1-268:head# 2020-09-12T14:34:31.711 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.744 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:0fabf252:::obj-1-441:head# 2020-09-12T14:34:31.745 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.778 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:0fb1697a:::obj-1-443:head# 2020-09-12T14:34:31.778 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.811 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:105d3efe:::obj-1-490:head# 2020-09-12T14:34:31.811 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.844 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:10f52c01:::obj-1-449:head# 2020-09-12T14:34:31.845 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.878 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:11965425:::obj-1-335:head# 2020-09-12T14:34:31.878 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.911 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:11c9f5f8:::obj-1-565:head# 2020-09-12T14:34:31.912 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.945 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:11cfcfb6:::obj-1-588:head# 2020-09-12T14:34:31.945 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:31.978 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:11d46f0a:::obj-1-387:head# 2020-09-12T14:34:31.978 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.012 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:11ebb0d6:::obj-1-214:head# 2020-09-12T14:34:32.012 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.045 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:1295e402:::obj-1-178:head# 2020-09-12T14:34:32.046 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.079 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:12c996c1:::obj-1-517:head# 2020-09-12T14:34:32.079 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.112 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:133dec15:::obj-1-475:head# 2020-09-12T14:34:32.112 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.146 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:1475c981:::obj-1-505:head# 2020-09-12T14:34:32.146 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.179 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:148e92a4:::obj-1-509:head# 2020-09-12T14:34:32.179 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.212 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:1490b121:::obj-1-172:head# 2020-09-12T14:34:32.213 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.246 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:14ce082c:::obj-1-99:head# 2020-09-12T14:34:32.246 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.279 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:1552ad99:::obj-1-41:head# 2020-09-12T14:34:32.280 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.313 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:15544270:::obj-1-271:head# 2020-09-12T14:34:32.313 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.372 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:1589d957:::obj-1-86:head# 2020-09-12T14:34:32.372 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.422 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:16db3947:::obj-1-477:head# 2020-09-12T14:34:32.422 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.455 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:16dc98bd:::obj-1-375:head# 2020-09-12T14:34:32.456 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.481 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:17e54db3:::obj-1-196:head# 2020-09-12T14:34:32.481 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.506 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:17f2ad3f:::obj-1-10:head# 2020-09-12T14:34:32.506 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.531 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:18075e41:::obj-1-55:head# 2020-09-12T14:34:32.531 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.556 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:18086253:::obj-1-71:head# 2020-09-12T14:34:32.556 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.581 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:191cf8d8:::obj-1-372:head# 2020-09-12T14:34:32.581 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.640 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:19311f70:::obj-1-454:head# 2020-09-12T14:34:32.640 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.682 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:198318e4:::obj-1-464:head# 2020-09-12T14:34:32.682 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.707 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:1999d806:::obj-1-163:head# 2020-09-12T14:34:32.707 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.740 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:1a92cab7:::obj-1-412:head# 2020-09-12T14:34:32.741 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.765 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:1b16c0f6:::obj-1-210:head# 2020-09-12T14:34:32.766 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.790 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:1b53ed46:::obj-1-577:head# 2020-09-12T14:34:32.791 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.824 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:1b5c5a8d:::obj-1-556:head# 2020-09-12T14:34:32.824 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.857 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:1c12c05e:::obj-1-79:head# 2020-09-12T14:34:32.858 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.882 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:1c571ba5:::obj-1-262:head# 2020-09-12T14:34:32.883 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.908 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:1c5f42b7:::obj-1-92:head# 2020-09-12T14:34:32.908 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.933 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:1c6ad881:::obj-1-312:head# 2020-09-12T14:34:32.933 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:32.958 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:1c89877b:::obj-1-142:head# 2020-09-12T14:34:32.958 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.000 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:1e4168ca:::obj-1-282:head# 2020-09-12T14:34:33.000 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.025 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:1e424445:::obj-1-484:head# 2020-09-12T14:34:33.025 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.050 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:1e467959:::obj-1-22:head# 2020-09-12T14:34:33.050 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.075 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:1eaf7e7c:::obj-1-392:head# 2020-09-12T14:34:33.075 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.100 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:1f06c86d:::obj-1-307:head# 2020-09-12T14:34:33.100 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.125 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:1f2ee1f8:::obj-1-306:head# 2020-09-12T14:34:33.126 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.151 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:1f4ef040:::obj-1-349:head# 2020-09-12T14:34:33.151 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.176 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:2039481f:::obj-1-545:head# 2020-09-12T14:34:33.176 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.201 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:22233df0:::obj-1-543:head# 2020-09-12T14:34:33.201 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.226 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:2225184e:::obj-1-266:head# 2020-09-12T14:34:33.226 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.251 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:234766bb:::obj-1-281:head# 2020-09-12T14:34:33.251 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.276 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:2372e022:::obj-1-205:head# 2020-09-12T14:34:33.276 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.301 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:2379547b:::obj-1-113:head# 2020-09-12T14:34:33.302 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.326 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:23ae14bf:::obj-1-531:head# 2020-09-12T14:34:33.327 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.352 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:23bc5f90:::obj-1-452:head# 2020-09-12T14:34:33.352 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.377 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:23fbef3d:::obj-1-82:head# 2020-09-12T14:34:33.377 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.402 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:24106c2b:::obj-1-232:head# 2020-09-12T14:34:33.402 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.427 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:2529b4e4:::obj-1-75:head# 2020-09-12T14:34:33.427 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.452 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:25a24ae7:::obj-1-459:head# 2020-09-12T14:34:33.453 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.477 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:26627aa9:::obj-1-7:head# 2020-09-12T14:34:33.478 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.502 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:26b77221:::obj-1-227:head# 2020-09-12T14:34:33.503 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.536 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:27450826:::obj-1-344:head# 2020-09-12T14:34:33.536 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.561 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:278905af:::obj-1-592:head# 2020-09-12T14:34:33.561 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.586 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:27ad0d2d:::obj-1-136:head# 2020-09-12T14:34:33.586 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.620 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:28a8c213:::obj-1-118:head# 2020-09-12T14:34:33.620 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.653 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:28c0c009:::obj-1-74:head# 2020-09-12T14:34:33.653 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.687 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:2a214708:::obj-1-260:head# 2020-09-12T14:34:33.687 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.720 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:2a67e195:::obj-1-474:head# 2020-09-12T14:34:33.720 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.753 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:2aa852ff:::obj-1-5:head# 2020-09-12T14:34:33.754 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.787 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:2ba1fb52:::obj-1-563:head# 2020-09-12T14:34:33.787 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.820 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:2bae8cb9:::obj-1-320:head# 2020-09-12T14:34:33.821 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.854 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:2c569d02:::obj-1-139:head# 2020-09-12T14:34:33.854 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.887 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:2c71d691:::obj-1-201:head# 2020-09-12T14:34:33.888 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.929 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:2c88e92c:::obj-1-389:head# 2020-09-12T14:34:33.929 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:33.971 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:2d659109:::obj-1-580:head# 2020-09-12T14:34:33.971 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.004 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:2d74ea60:::obj-1-230:head# 2020-09-12T14:34:34.005 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.038 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:2de96edf:::obj-1-233:head# 2020-09-12T14:34:34.039 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.080 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:2e43da40:::obj-1-382:head# 2020-09-12T14:34:34.080 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.113 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:2e530c9f:::obj-1-358:head# 2020-09-12T14:34:34.113 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.146 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:2ed02239:::obj-1-81:head# 2020-09-12T14:34:34.147 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.188 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:30264a0c:::obj-1-373:head# 2020-09-12T14:34:34.189 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.239 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:302ca5e7:::obj-1-206:head# 2020-09-12T14:34:34.239 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.272 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:309878e8:::obj-1-508:head# 2020-09-12T14:34:34.272 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.305 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:309a0ac3:::obj-1-188:head# 2020-09-12T14:34:34.306 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.364 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:30f030e9:::obj-1-53:head# 2020-09-12T14:34:34.365 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.432 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:31242a73:::obj-1-528:head# 2020-09-12T14:34:34.432 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.490 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:31463825:::obj-1-34:head# 2020-09-12T14:34:34.490 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.531 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3166dc86:::obj-1-355:head# 2020-09-12T14:34:34.532 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.573 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:31baa2fa:::obj-1-252:head# 2020-09-12T14:34:34.574 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.615 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:31d8935e:::obj-1-193:head# 2020-09-12T14:34:34.616 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.674 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:32670b14:::obj-1-305:head# 2020-09-12T14:34:34.674 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.707 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:33da02fa:::obj-1-114:head# 2020-09-12T14:34:34.708 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.741 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:34373133:::obj-1-446:head# 2020-09-12T14:34:34.741 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.774 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:347b7083:::obj-1-89:head# 2020-09-12T14:34:34.775 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.808 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:349125c1:::obj-1-148:head# 2020-09-12T14:34:34.808 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.833 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:34b80eee:::obj-1-138:head# 2020-09-12T14:34:34.833 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.858 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:34db48f8:::obj-1-49:head# 2020-09-12T14:34:34.858 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.891 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3669ff0c:::obj-1-313:head# 2020-09-12T14:34:34.892 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.925 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:36c1b610:::obj-1-249:head# 2020-09-12T14:34:34.925 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.958 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:374c250b:::obj-1-324:head# 2020-09-12T14:34:34.959 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:34.992 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3761a149:::obj-1-342:head# 2020-09-12T14:34:34.992 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.025 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:379ff95a:::obj-1-341:head# 2020-09-12T14:34:35.025 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.058 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:37dc15a6:::obj-1-26:head# 2020-09-12T14:34:35.059 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.117 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:38130e33:::obj-1-552:head# 2020-09-12T14:34:35.118 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.167 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:39c44880:::obj-1-538:head# 2020-09-12T14:34:35.168 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.201 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3a08dec9:::obj-1-368:head# 2020-09-12T14:34:35.201 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.234 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3a2b46a6:::obj-1-407:head# 2020-09-12T14:34:35.235 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.268 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3a7dfe0a:::obj-1-285:head# 2020-09-12T14:34:35.268 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.301 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3aebb144:::obj-1-248:head# 2020-09-12T14:34:35.302 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.335 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3b08b36f:::obj-1-310:head# 2020-09-12T14:34:35.335 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.368 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3b28ffa3:::obj-1-295:head# 2020-09-12T14:34:35.368 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.402 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3ba79bd1:::obj-1-444:head# 2020-09-12T14:34:35.402 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.435 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3c05df8d:::obj-1-319:head# 2020-09-12T14:34:35.435 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.469 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3c57e6a9:::obj-1-436:head# 2020-09-12T14:34:35.469 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.502 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3c73d8ee:::obj-1-298:head# 2020-09-12T14:34:35.502 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.535 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3cb6ae2a:::obj-1-400:head# 2020-09-12T14:34:35.536 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.569 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3cb71491:::obj-1-328:head# 2020-09-12T14:34:35.569 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.603 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3cb842fe:::obj-1-541:head# 2020-09-12T14:34:35.603 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.636 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3cf6835e:::obj-1-496:head# 2020-09-12T14:34:35.636 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.669 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3cf78507:::obj-1-573:head# 2020-09-12T14:34:35.670 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.703 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3d0dab84:::obj-1-554:head# 2020-09-12T14:34:35.703 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.736 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3d1a5f9d:::obj-1-323:head# 2020-09-12T14:34:35.737 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.770 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3da5f3a2:::obj-1-414:head# 2020-09-12T14:34:35.770 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.828 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3dbb65f0:::obj-1-440:head# 2020-09-12T14:34:35.828 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.862 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3dc94618:::obj-1-223:head# 2020-09-12T14:34:35.862 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.895 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3df3d5ab:::obj-1-25:head# 2020-09-12T14:34:35.895 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.928 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3e167988:::obj-1-336:head# 2020-09-12T14:34:35.929 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.962 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3e4ef3bd:::obj-1-104:head# 2020-09-12T14:34:35.962 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:35.995 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3e9e1efd:::obj-1-357:head# 2020-09-12T14:34:35.996 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.029 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3ebbe3a7:::obj-1-42:head# 2020-09-12T14:34:36.029 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.062 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3ec80373:::obj-1-403:head# 2020-09-12T14:34:36.063 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.096 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3ed086e4:::obj-1-13:head# 2020-09-12T14:34:36.096 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.129 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:3fe8e157:::obj-1-164:head# 2020-09-12T14:34:36.129 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.163 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:401ddfd4:::obj-1-379:head# 2020-09-12T14:34:36.163 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.196 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:4092f6cd:::obj-1-430:head# 2020-09-12T14:34:36.196 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.230 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:414bd7cc:::obj-1-562:head# 2020-09-12T14:34:36.230 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.263 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:422bd9d0:::obj-1-287:head# 2020-09-12T14:34:36.263 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.297 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:4255002d:::obj-1-578:head# 2020-09-12T14:34:36.297 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.340 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:4270725c:::obj-1-109:head# 2020-09-12T14:34:36.341 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.414 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:42836669:::obj-1-31:head# 2020-09-12T14:34:36.414 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.447 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:428d93ef:::obj-1-32:head# 2020-09-12T14:34:36.448 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.483 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:4327ef5b:::obj-1-453:head# 2020-09-12T14:34:36.484 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.523 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:440eb07b:::obj-1-236:head# 2020-09-12T14:34:36.523 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.556 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:442a028a:::obj-1-365:head# 2020-09-12T14:34:36.556 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.606 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:449ca54a:::obj-1-161:head# 2020-09-12T14:34:36.607 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.690 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:45128c3b:::obj-1-343:head# 2020-09-12T14:34:36.690 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.740 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:45801bf6:::obj-1-264:head# 2020-09-12T14:34:36.741 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.791 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:45896f61:::obj-1-48:head# 2020-09-12T14:34:36.791 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.841 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:459b7f1c:::obj-1-190:head# 2020-09-12T14:34:36.841 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.883 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:466af96a:::obj-1-486:head# 2020-09-12T14:34:36.883 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.933 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:47387ce4:::obj-1-575:head# 2020-09-12T14:34:36.933 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:36.983 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:47c94cc8:::obj-1-564:head# 2020-09-12T14:34:36.984 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.025 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:47d7f8ed:::obj-1-544:head# 2020-09-12T14:34:37.026 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.075 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:47fc0ec3:::obj-1-558:head# 2020-09-12T14:34:37.076 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.126 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:48ba42d8:::obj-1-156:head# 2020-09-12T14:34:37.126 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.176 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:494e269e:::obj-1-600:head# 2020-09-12T14:34:37.176 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.210 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:49646095:::obj-1-73:head# 2020-09-12T14:34:37.210 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.243 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:49eb65ab:::obj-1-374:head# 2020-09-12T14:34:37.243 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.276 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:4a0075a4:::obj-1-432:head# 2020-09-12T14:34:37.277 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.310 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:4af1ffc2:::obj-1-72:head# 2020-09-12T14:34:37.310 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.343 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:4bc1c46f:::obj-1-346:head# 2020-09-12T14:34:37.344 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.377 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:4bd55e68:::obj-1-93:head# 2020-09-12T14:34:37.377 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.410 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:4c6d2760:::obj-1-198:head# 2020-09-12T14:34:37.411 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.444 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:4cb9be3b:::obj-1-451:head# 2020-09-12T14:34:37.444 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.477 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:4d260a64:::obj-1-514:head# 2020-09-12T14:34:37.478 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.511 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:4d7d6284:::obj-1-583:head# 2020-09-12T14:34:37.511 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.544 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:4e79ac02:::obj-1-437:head# 2020-09-12T14:34:37.545 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.578 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:4e980099:::obj-1-507:head# 2020-09-12T14:34:37.578 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.628 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:4ea8f99c:::obj-1-211:head# 2020-09-12T14:34:37.628 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.661 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:4eabdc2a:::obj-1-590:head# 2020-09-12T14:34:37.662 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.712 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:4f97fd73:::obj-1-243:head# 2020-09-12T14:34:37.712 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.762 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:4fe00224:::obj-1-229:head# 2020-09-12T14:34:37.762 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.812 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:4fead9ab:::obj-1-476:head# 2020-09-12T14:34:37.812 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.862 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:4ffddfd7:::obj-1-462:head# 2020-09-12T14:34:37.863 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.913 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:5085c04d:::obj-1-100:head# 2020-09-12T14:34:37.913 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:37.963 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:50a8b465:::obj-1-574:head# 2020-09-12T14:34:37.963 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:38.013 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:518a151f:::obj-1-189:head# 2020-09-12T14:34:38.013 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:38.063 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:519afc95:::obj-1-314:head# 2020-09-12T14:34:38.064 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:38.116 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:52199781:::obj-1-202:head# 2020-09-12T14:34:38.116 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:38.166 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:52313062:::obj-1-337:head# 2020-09-12T14:34:38.166 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:38.225 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:526a5196:::obj-1-225:head# 2020-09-12T14:34:38.225 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:38.275 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:53128d32:::obj-1-483:head# 2020-09-12T14:34:38.275 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:38.333 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:53c2bb21:::obj-1-532:head# 2020-09-12T14:34:38.334 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:38.400 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:53fa4f3c:::obj-1-423:head# 2020-09-12T14:34:38.401 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:38.426 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:541c1d30:::obj-1-143:head# 2020-09-12T14:34:38.426 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:38.451 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:548d20c3:::obj-1-352:head# 2020-09-12T14:34:38.451 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:38.476 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:54e4c6f9:::obj-1-101:head# 2020-09-12T14:34:38.476 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:38.501 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:55016e06:::obj-1-276:head# 2020-09-12T14:34:38.501 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:38.526 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:5510c784:::obj-1-491:head# 2020-09-12T14:34:38.526 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:38.577 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:551275ef:::obj-1-239:head# 2020-09-12T14:34:38.577 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:38.657 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:556df001:::obj-1-46:head# 2020-09-12T14:34:38.658 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:38.727 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:558fb5b1:::obj-1-536:head# 2020-09-12T14:34:38.727 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:38.777 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:56390d98:::obj-1-64:head# 2020-09-12T14:34:38.778 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:38.828 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:564f962a:::obj-1-125:head# 2020-09-12T14:34:38.828 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:38.878 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:56936e94:::obj-1-251:head# 2020-09-12T14:34:38.878 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:38.928 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:57585d3d:::obj-1-294:head# 2020-09-12T14:34:38.929 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:38.978 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:5764cf7b:::obj-1-479:head# 2020-09-12T14:34:38.979 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:39.012 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:5782341a:::obj-1-218:head# 2020-09-12T14:34:39.012 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:39.045 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:57d07a1a:::obj-1-317:head# 2020-09-12T14:34:39.046 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:39.096 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:58700205:::obj-1-176:head# 2020-09-12T14:34:39.096 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:39.146 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:58aee718:::obj-1-434:head# 2020-09-12T14:34:39.146 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:39.196 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:594c9d52:::obj-1-458:head# 2020-09-12T14:34:39.196 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:39.246 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:5a70b555:::obj-1-279:head# 2020-09-12T14:34:39.247 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:39.297 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:5b02568b:::obj-1-586:head# 2020-09-12T14:34:39.297 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:39.347 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:5b69cdcf:::obj-1-391:head# 2020-09-12T14:34:39.347 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:39.397 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:5cf151b5:::obj-1-510:head# 2020-09-12T14:34:39.397 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:39.447 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:5d7fd07f:::obj-1-152:head# 2020-09-12T14:34:39.448 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:39.498 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:5de288d7:::obj-1-235:head# 2020-09-12T14:34:39.498 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:39.548 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:5ec497cd:::obj-1-44:head# 2020-09-12T14:34:39.548 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:39.598 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:5fb3a09f:::obj-1-518:head# 2020-09-12T14:34:39.598 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:39.648 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:5ff3c2ef:::obj-1-470:head# 2020-09-12T14:34:39.649 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:39.707 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:601efcb1:::obj-1-549:head# 2020-09-12T14:34:39.708 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:39.758 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:60831338:::obj-1-263:head# 2020-09-12T14:34:39.758 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:39.808 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:6092777b:::obj-1-145:head# 2020-09-12T14:34:39.808 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:39.858 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:60b17206:::obj-1-411:head# 2020-09-12T14:34:39.858 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:39.908 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:610c957c:::obj-1-278:head# 2020-09-12T14:34:39.909 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:39.958 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:613b5ef4:::obj-1-504:head# 2020-09-12T14:34:39.959 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:40.009 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:61ed7d60:::obj-1-274:head# 2020-09-12T14:34:40.009 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:40.059 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:621b6751:::obj-1-212:head# 2020-09-12T14:34:40.059 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:40.109 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:62b0aba9:::obj-1-253:head# 2020-09-12T14:34:40.109 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:40.343 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:64017240:::obj-1-102:head# 2020-09-12T14:34:40.344 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:40.435 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:645f4f49:::obj-1-332:head# 2020-09-12T14:34:40.436 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:40.485 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:65db82b9:::obj-1-334:head# 2020-09-12T14:34:40.486 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:40.536 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:667422e5:::obj-1-465:head# 2020-09-12T14:34:40.536 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:40.586 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:66c50bc2:::obj-1-322:head# 2020-09-12T14:34:40.586 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:40.653 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:681a4577:::obj-1-511:head# 2020-09-12T14:34:40.653 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:40.703 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:6874e007:::obj-1-398:head# 2020-09-12T14:34:40.704 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:40.754 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:693746c1:::obj-1-582:head# 2020-09-12T14:34:40.754 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:40.804 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:69638b0c:::obj-1-8:head# 2020-09-12T14:34:40.804 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:40.854 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:6967de6e:::obj-1-331:head# 2020-09-12T14:34:40.854 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:40.890 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:69da97aa:::obj-1-340:head# 2020-09-12T14:34:40.890 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:40.923 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:6b3a50a6:::obj-1-144:head# 2020-09-12T14:34:40.923 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:40.965 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:6be4ef26:::obj-1-579:head# 2020-09-12T14:34:40.965 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:40.998 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:6d735549:::obj-1-599:head# 2020-09-12T14:34:40.998 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.031 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:6e22c25e:::obj-1-9:head# 2020-09-12T14:34:41.032 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.065 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:6e2ced4e:::obj-1-370:head# 2020-09-12T14:34:41.065 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.098 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:6e961169:::obj-1-428:head# 2020-09-12T14:34:41.098 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.133 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:6efcbbee:::obj-1-168:head# 2020-09-12T14:34:41.133 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.165 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:6f20741c:::obj-1-33:head# 2020-09-12T14:34:41.165 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.198 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:6fe4c885:::obj-1-273:head# 2020-09-12T14:34:41.199 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.232 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:6fffdea2:::obj-1-425:head# 2020-09-12T14:34:41.232 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.265 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:70716852:::obj-1-593:head# 2020-09-12T14:34:41.265 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.298 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:70919606:::obj-1-515:head# 2020-09-12T14:34:41.299 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.332 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:70c39d76:::obj-1-467:head# 2020-09-12T14:34:41.332 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.365 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:70f57ad3:::obj-1-361:head# 2020-09-12T14:34:41.365 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.398 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:7177c7b5:::obj-1-503:head# 2020-09-12T14:34:41.399 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.432 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:719b69bd:::obj-1-497:head# 2020-09-12T14:34:41.432 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.465 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:724f9347:::obj-1-6:head# 2020-09-12T14:34:41.465 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.498 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:726d9f6b:::obj-1-111:head# 2020-09-12T14:34:41.499 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.540 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:72a91baf:::obj-1-439:head# 2020-09-12T14:34:41.541 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.574 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:72aa43ee:::obj-1-534:head# 2020-09-12T14:34:41.574 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.607 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:72c40e11:::obj-1-288:head# 2020-09-12T14:34:41.607 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.640 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:74196d45:::obj-1-246:head# 2020-09-12T14:34:41.641 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.674 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:743f42b1:::obj-1-326:head# 2020-09-12T14:34:41.674 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.707 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:7523b538:::obj-1-494:head# 2020-09-12T14:34:41.708 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.741 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:75310ed6:::obj-1-364:head# 2020-09-12T14:34:41.741 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.774 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:7568e0bc:::obj-1-585:head# 2020-09-12T14:34:41.774 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.807 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:760219a3:::obj-1-353:head# 2020-09-12T14:34:41.808 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.841 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:763338c9:::obj-1-555:head# 2020-09-12T14:34:41.841 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.866 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:77bc2d65:::obj-1-386:head# 2020-09-12T14:34:41.866 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.891 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:77ed0c7a:::obj-1-492:head# 2020-09-12T14:34:41.891 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.916 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:785a4052:::obj-1-456:head# 2020-09-12T14:34:41.916 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.941 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:796799c3:::obj-1-195:head# 2020-09-12T14:34:41.941 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.966 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:79caf0c1:::obj-1-445:head# 2020-09-12T14:34:41.966 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:41.991 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:7af5b3e4:::obj-1-217:head# 2020-09-12T14:34:41.991 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.015 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:7b3a9063:::obj-1-169:head# 2020-09-12T14:34:42.016 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.041 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:7b509a92:::obj-1-561:head# 2020-09-12T14:34:42.041 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.066 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:7bc646b1:::obj-1-415:head# 2020-09-12T14:34:42.066 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.091 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:7c2f9188:::obj-1-482:head# 2020-09-12T14:34:42.091 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.116 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:7d0da936:::obj-1-11:head# 2020-09-12T14:34:42.116 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.141 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:7d25ca2d:::obj-1-242:head# 2020-09-12T14:34:42.141 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.166 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:7d64f81e:::obj-1-473:head# 2020-09-12T14:34:42.166 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.191 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:7ef2ea1b:::obj-1-417:head# 2020-09-12T14:34:42.191 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.216 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:7f52b238:::obj-1-308:head# 2020-09-12T14:34:42.216 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.241 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:7f7a313c:::obj-1-551:head# 2020-09-12T14:34:42.241 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.266 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:806827a7:::obj-1-472:head# 2020-09-12T14:34:42.266 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.291 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:8112c3ab:::obj-1-45:head# 2020-09-12T14:34:42.291 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.316 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:81365b55:::obj-1-315:head# 2020-09-12T14:34:42.316 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.369 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:818ffdc3:::obj-1-345:head# 2020-09-12T14:34:42.369 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.416 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:81babab8:::obj-1-535:head# 2020-09-12T14:34:42.416 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.441 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:8288f652:::obj-1-347:head# 2020-09-12T14:34:42.441 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.466 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:8312eadd:::obj-1-14:head# 2020-09-12T14:34:42.466 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.491 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:83b89bc7:::obj-1-137:head# 2020-09-12T14:34:42.491 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.524 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:84a12321:::obj-1-134:head# 2020-09-12T14:34:42.525 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.558 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:84b26d62:::obj-1-408:head# 2020-09-12T14:34:42.558 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.591 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:84bbb26e:::obj-1-43:head# 2020-09-12T14:34:42.591 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.671 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:86fc0014:::obj-1-222:head# 2020-09-12T14:34:42.671 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.708 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:87488bf4:::obj-1-174:head# 2020-09-12T14:34:42.708 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.741 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:8754389a:::obj-1-65:head# 2020-09-12T14:34:42.742 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.775 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:87a6ad67:::obj-1-366:head# 2020-09-12T14:34:42.775 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.808 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:87d32b83:::obj-1-513:head# 2020-09-12T14:34:42.808 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.841 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:8813a346:::obj-1-540:head# 2020-09-12T14:34:42.842 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.874 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:88146951:::obj-1-216:head# 2020-09-12T14:34:42.875 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.908 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:8820800d:::obj-1-83:head# 2020-09-12T14:34:42.908 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.941 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:88319745:::obj-1-98:head# 2020-09-12T14:34:42.942 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:42.975 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:88c969b8:::obj-1-54:head# 2020-09-12T14:34:42.975 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.016 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:895a2d53:::obj-1-150:head# 2020-09-12T14:34:43.017 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.050 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:8ae006a0:::obj-1-241:head# 2020-09-12T14:34:43.050 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.083 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:8bd44e64:::obj-1-300:head# 2020-09-12T14:34:43.083 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.117 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:8beb257f:::obj-1-3:head# 2020-09-12T14:34:43.117 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.150 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:8cd8c743:::obj-1-527:head# 2020-09-12T14:34:43.150 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.183 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:8daa99c7:::obj-1-237:head# 2020-09-12T14:34:43.183 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.216 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:8f5bccbf:::obj-1-59:head# 2020-09-12T14:34:43.217 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.250 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:8f6d9945:::obj-1-151:head# 2020-09-12T14:34:43.250 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.283 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:8f7c568c:::obj-1-261:head# 2020-09-12T14:34:43.283 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.316 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:8fbc59a9:::obj-1-589:head# 2020-09-12T14:34:43.317 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.350 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:90225d94:::obj-1-194:head# 2020-09-12T14:34:43.350 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.383 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:904c50d3:::obj-1-63:head# 2020-09-12T14:34:43.383 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.416 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:90e1bd79:::obj-1-450:head# 2020-09-12T14:34:43.417 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.458 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:9274b06d:::obj-1-460:head# 2020-09-12T14:34:43.458 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.492 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:92af41f5:::obj-1-409:head# 2020-09-12T14:34:43.492 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.525 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:935a08d6:::obj-1-192:head# 2020-09-12T14:34:43.525 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.558 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:935f8a13:::obj-1-24:head# 2020-09-12T14:34:43.558 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.592 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:95230148:::obj-1-280:head# 2020-09-12T14:34:43.592 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.625 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:95342eb0:::obj-1-390:head# 2020-09-12T14:34:43.625 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.658 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:9659d09c:::obj-1-292:head# 2020-09-12T14:34:43.659 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.692 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:970e8cb4:::obj-1-18:head# 2020-09-12T14:34:43.692 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.725 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:97d21cf1:::obj-1-80:head# 2020-09-12T14:34:43.725 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.758 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:982bdb12:::obj-1-302:head# 2020-09-12T14:34:43.759 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.792 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:985dfd75:::obj-1-91:head# 2020-09-12T14:34:43.792 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.825 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:986d245a:::obj-1-154:head# 2020-09-12T14:34:43.825 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.867 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:98a03e99:::obj-1-529:head# 2020-09-12T14:34:43.867 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.900 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:98b7103e:::obj-1-275:head# 2020-09-12T14:34:43.900 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.933 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:990f0ecf:::obj-1-427:head# 2020-09-12T14:34:43.934 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:43.967 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:996c1f26:::obj-1-96:head# 2020-09-12T14:34:43.967 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.000 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:99bd2018:::obj-1-17:head# 2020-09-12T14:34:44.000 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.033 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:9a7434c9:::obj-1-469:head# 2020-09-12T14:34:44.034 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.067 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:9a8e36cc:::obj-1-598:head# 2020-09-12T14:34:44.067 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.100 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:9abcc369:::obj-1-442:head# 2020-09-12T14:34:44.100 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.142 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:9b0ea06e:::obj-1-231:head# 2020-09-12T14:34:44.142 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.175 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:9b34e80d:::obj-1-378:head# 2020-09-12T14:34:44.176 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.209 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:9b55a2a9:::obj-1-60:head# 2020-09-12T14:34:44.209 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.242 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:9be3f2da:::obj-1-597:head# 2020-09-12T14:34:44.242 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.275 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:9bf62c88:::obj-1-447:head# 2020-09-12T14:34:44.276 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.309 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:9c31857b:::obj-1-272:head# 2020-09-12T14:34:44.309 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.369 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:9c533e91:::obj-1-19:head# 2020-09-12T14:34:44.369 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.425 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:9ca70688:::obj-1-570:head# 2020-09-12T14:34:44.426 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.459 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:9ccce337:::obj-1-147:head# 2020-09-12T14:34:44.459 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.492 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:9d4b2dfc:::obj-1-267:head# 2020-09-12T14:34:44.492 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.525 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:9d4df969:::obj-1-286:head# 2020-09-12T14:34:44.526 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.559 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:9d714651:::obj-1-495:head# 2020-09-12T14:34:44.559 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.592 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:9da65518:::obj-1-498:head# 2020-09-12T14:34:44.592 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.673 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:9e94f27b:::obj-1-209:head# 2020-09-12T14:34:44.674 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.709 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:9f39e910:::obj-1-354:head# 2020-09-12T14:34:44.709 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.742 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:9f8f9ef4:::obj-1-226:head# 2020-09-12T14:34:44.743 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.776 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:9fcd3f36:::obj-1-47:head# 2020-09-12T14:34:44.776 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.809 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a0326d7d:::obj-1-338:head# 2020-09-12T14:34:44.809 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.842 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a05e7475:::obj-1-62:head# 2020-09-12T14:34:44.843 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.876 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a0955052:::obj-1-533:head# 2020-09-12T14:34:44.876 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.909 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a0aa6848:::obj-1-38:head# 2020-09-12T14:34:44.909 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.942 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a0fdc285:::obj-1-133:head# 2020-09-12T14:34:44.943 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:44.976 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a10290b8:::obj-1-245:head# 2020-09-12T14:34:44.976 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.009 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a118a8a8:::obj-1-500:head# 2020-09-12T14:34:45.009 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.042 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a11a4079:::obj-1-199:head# 2020-09-12T14:34:45.043 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.075 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a1b3ec36:::obj-1-15:head# 2020-09-12T14:34:45.076 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.109 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a1f5079b:::obj-1-468:head# 2020-09-12T14:34:45.109 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.157 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a1ff04b5:::obj-1-167:head# 2020-09-12T14:34:45.158 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.209 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a21cc437:::obj-1-128:head# 2020-09-12T14:34:45.209 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.284 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a2966c77:::obj-1-140:head# 2020-09-12T14:34:45.285 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.318 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a321e537:::obj-1-184:head# 2020-09-12T14:34:45.318 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.351 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a33418d8:::obj-1-488:head# 2020-09-12T14:34:45.351 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.384 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a38b1c1a:::obj-1-584:head# 2020-09-12T14:34:45.384 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.418 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a3f83221:::obj-1-158:head# 2020-09-12T14:34:45.418 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.451 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a46e33eb:::obj-1-106:head# 2020-09-12T14:34:45.451 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.484 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a491460d:::obj-1-421:head# 2020-09-12T14:34:45.485 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.517 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a4dd813b:::obj-1-530:head# 2020-09-12T14:34:45.518 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.551 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a4e0d96d:::obj-1-351:head# 2020-09-12T14:34:45.551 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.584 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a52657b4:::obj-1-557:head# 2020-09-12T14:34:45.584 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.617 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a5b3bca6:::obj-1-277:head# 2020-09-12T14:34:45.618 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.651 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a5f5a9fb:::obj-1-170:head# 2020-09-12T14:34:45.651 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.684 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a6aeed57:::obj-1-141:head# 2020-09-12T14:34:45.684 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.717 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a6dda70f:::obj-1-471:head# 2020-09-12T14:34:45.718 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.751 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a71186c0:::obj-1-406:head# 2020-09-12T14:34:45.751 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.784 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a764ad8c:::obj-1-221:head# 2020-09-12T14:34:45.784 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.818 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a7b63473:::obj-1-131:head# 2020-09-12T14:34:45.818 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.851 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a88042fa:::obj-1-296:head# 2020-09-12T14:34:45.851 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.884 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:a8805fe0:::obj-1-213:head# 2020-09-12T14:34:45.885 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.943 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:aa4f12b1:::obj-1-270:head# 2020-09-12T14:34:45.943 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:45.976 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:ab27f701:::obj-1-399:head# 2020-09-12T14:34:45.976 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.009 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:ab8027c1:::obj-1-360:head# 2020-09-12T14:34:46.010 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.043 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:ac3ef8ab:::obj-1-371:head# 2020-09-12T14:34:46.043 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.076 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:ac4db76d:::obj-1-405:head# 2020-09-12T14:34:46.076 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.109 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:ad53cb29:::obj-1-256:head# 2020-09-12T14:34:46.109 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.143 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:ad704c16:::obj-1-485:head# 2020-09-12T14:34:46.143 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.176 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:adec4920:::obj-1-107:head# 2020-09-12T14:34:46.176 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.209 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:ae3a94b3:::obj-1-95:head# 2020-09-12T14:34:46.210 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.243 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:ae92616e:::obj-1-448:head# 2020-09-12T14:34:46.243 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.276 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:ae975168:::obj-1-426:head# 2020-09-12T14:34:46.276 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.309 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:aee8fd62:::obj-1-27:head# 2020-09-12T14:34:46.310 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.361 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:af38aa87:::obj-1-50:head# 2020-09-12T14:34:46.361 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.418 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:b0327eb3:::obj-1-380:head# 2020-09-12T14:34:46.418 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.451 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:b03cd166:::obj-1-203:head# 2020-09-12T14:34:46.452 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.485 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:b06f06e0:::obj-1-126:head# 2020-09-12T14:34:46.485 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.518 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:b0972538:::obj-1-506:head# 2020-09-12T14:34:46.518 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.551 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:b0cfb86a:::obj-1-359:head# 2020-09-12T14:34:46.551 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.585 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:b10adb1c:::obj-1-333:head# 2020-09-12T14:34:46.585 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.637 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:b20b9004:::obj-1-401:head# 2020-09-12T14:34:46.637 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.685 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:b2113757:::obj-1-4:head# 2020-09-12T14:34:46.685 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.718 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:b22441e9:::obj-1-29:head# 2020-09-12T14:34:46.718 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.751 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:b2472384:::obj-1-207:head# 2020-09-12T14:34:46.752 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.785 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:b2604227:::obj-1-502:head# 2020-09-12T14:34:46.785 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.818 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:b3277f89:::obj-1-175:head# 2020-09-12T14:34:46.818 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.851 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:b3d09e0d:::obj-1-258:head# 2020-09-12T14:34:46.852 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.885 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:b3e9969a:::obj-1-499:head# 2020-09-12T14:34:46.885 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.918 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:b4892e7d:::obj-1-155:head# 2020-09-12T14:34:46.918 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.951 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:b4b76084:::obj-1-581:head# 2020-09-12T14:34:46.952 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:46.985 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:b6e47ae5:::obj-1-215:head# 2020-09-12T14:34:46.985 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.018 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:b75af5ff:::obj-1-228:head# 2020-09-12T14:34:47.018 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.051 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:b8e2e20f:::obj-1-220:head# 2020-09-12T14:34:47.052 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.093 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:b8e80c4e:::obj-1-576:head# 2020-09-12T14:34:47.093 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.135 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:ba502aee:::obj-1-255:head# 2020-09-12T14:34:47.135 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.176 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:ba9c3e6e:::obj-1-57:head# 2020-09-12T14:34:47.177 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.218 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:bb068062:::obj-1-461:head# 2020-09-12T14:34:47.219 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.260 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:bb16c115:::obj-1-157:head# 2020-09-12T14:34:47.260 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.302 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:bb447914:::obj-1-481:head# 2020-09-12T14:34:47.302 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.343 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:bb6c8e0f:::obj-1-348:head# 2020-09-12T14:34:47.343 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.385 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:bb77d6d0:::obj-1-301:head# 2020-09-12T14:34:47.385 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.427 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:bc51c5f6:::obj-1-422:head# 2020-09-12T14:34:47.427 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.468 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:bc838d5c:::obj-1-35:head# 2020-09-12T14:34:47.468 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.510 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:bceae5eb:::obj-1-56:head# 2020-09-12T14:34:47.510 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.552 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:bea17ef6:::obj-1-84:head# 2020-09-12T14:34:47.552 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.585 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:beb88be5:::obj-1-396:head# 2020-09-12T14:34:47.585 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.619 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:beed7b18:::obj-1-569:head# 2020-09-12T14:34:47.619 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.651 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:bf0bff49:::obj-1-385:head# 2020-09-12T14:34:47.652 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.685 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:bf545d5a:::obj-1-69:head# 2020-09-12T14:34:47.685 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.718 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:bf560a7f:::obj-1-40:head# 2020-09-12T14:34:47.718 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.752 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:bff6999c:::obj-1-240:head# 2020-09-12T14:34:47.752 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.785 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:c0a33d38:::obj-1-321:head# 2020-09-12T14:34:47.785 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.818 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:c1e4c9b4:::obj-1-299:head# 2020-09-12T14:34:47.818 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.852 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:c1ec515a:::obj-1-16:head# 2020-09-12T14:34:47.852 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.885 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:c22de3c8:::obj-1-110:head# 2020-09-12T14:34:47.885 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.918 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:c231cd7c:::obj-1-58:head# 2020-09-12T14:34:47.919 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.952 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:c2ee6ace:::obj-1-186:head# 2020-09-12T14:34:47.952 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:47.985 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:c32bfef2:::obj-1-181:head# 2020-09-12T14:34:47.985 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.018 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:c575a298:::obj-1-466:head# 2020-09-12T14:34:48.019 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.052 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:c58c3f80:::obj-1-493:head# 2020-09-12T14:34:48.052 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.085 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:c6442cf2:::obj-1-455:head# 2020-09-12T14:34:48.085 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.119 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:c6619413:::obj-1-37:head# 2020-09-12T14:34:48.119 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.152 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:c6c2f8e5:::obj-1-519:head# 2020-09-12T14:34:48.152 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.185 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:c6f811a2:::obj-1-180:head# 2020-09-12T14:34:48.185 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.219 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:c72af975:::obj-1-21:head# 2020-09-12T14:34:48.219 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.252 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:c73eb350:::obj-1-356:head# 2020-09-12T14:34:48.252 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.285 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:c7a683da:::obj-1-410:head# 2020-09-12T14:34:48.285 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.318 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:c7eb9bcc:::obj-1-553:head# 2020-09-12T14:34:48.319 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.393 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:c886df99:::obj-1-269:head# 2020-09-12T14:34:48.394 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.427 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:c932e3ec:::obj-1-135:head# 2020-09-12T14:34:48.427 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.460 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:c9b6507a:::obj-1-520:head# 2020-09-12T14:34:48.460 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.494 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:c9e3b064:::obj-1-244:head# 2020-09-12T14:34:48.494 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.527 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:cb705d66:::obj-1-377:head# 2020-09-12T14:34:48.527 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.560 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:cc80e25c:::obj-1-115:head# 2020-09-12T14:34:48.561 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.594 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:cd1ec88e:::obj-1-297:head# 2020-09-12T14:34:48.594 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.670 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:cd7e6936:::obj-1-234:head# 2020-09-12T14:34:48.670 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.702 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:ce32b69c:::obj-1-30:head# 2020-09-12T14:34:48.703 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.736 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:cf2b123c:::obj-1-293:head# 2020-09-12T14:34:48.736 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.769 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:cf2e80a9:::obj-1-542:head# 2020-09-12T14:34:48.769 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.802 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:cf51da6a:::obj-1-284:head# 2020-09-12T14:34:48.802 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.836 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:cf590f42:::obj-1-381:head# 2020-09-12T14:34:48.836 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.869 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:cf68a2b1:::obj-1-68:head# 2020-09-12T14:34:48.869 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.902 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:cf6aefb6:::obj-1-28:head# 2020-09-12T14:34:48.902 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.936 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:cfd8e6af:::obj-1-105:head# 2020-09-12T14:34:48.936 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:48.969 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d0640bad:::obj-1-254:head# 2020-09-12T14:34:48.969 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.002 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d101cda5:::obj-1-259:head# 2020-09-12T14:34:49.003 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.036 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d1be9386:::obj-1-416:head# 2020-09-12T14:34:49.036 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.077 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d243f036:::obj-1-265:head# 2020-09-12T14:34:49.078 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.111 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d2808c34:::obj-1-159:head# 2020-09-12T14:34:49.111 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.144 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d3415824:::obj-1-39:head# 2020-09-12T14:34:49.144 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.177 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d424f07a:::obj-1-547:head# 2020-09-12T14:34:49.178 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.211 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d44139f2:::obj-1-289:head# 2020-09-12T14:34:49.211 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.244 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d46015ab:::obj-1-546:head# 2020-09-12T14:34:49.244 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.277 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d4e503db:::obj-1-525:head# 2020-09-12T14:34:49.278 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.311 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d4e97d18:::obj-1-85:head# 2020-09-12T14:34:49.311 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.353 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d511492e:::obj-1-388:head# 2020-09-12T14:34:49.353 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.394 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d526a7fa:::obj-1-12:head# 2020-09-12T14:34:49.394 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.436 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d5667b59:::obj-1-369:head# 2020-09-12T14:34:49.436 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.478 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d57f2261:::obj-1-419:head# 2020-09-12T14:34:49.478 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.519 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d57f281b:::obj-1-250:head# 2020-09-12T14:34:49.520 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.561 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d6e05894:::obj-1-166:head# 2020-09-12T14:34:49.561 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.603 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d7689481:::obj-1-424:head# 2020-09-12T14:34:49.603 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.644 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d7d27743:::obj-1-129:head# 2020-09-12T14:34:49.645 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.686 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d7e4d3e9:::obj-1-127:head# 2020-09-12T14:34:49.686 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.736 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d7f1422a:::obj-1-594:head# 2020-09-12T14:34:49.736 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.778 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d8469fd2:::obj-1-153:head# 2020-09-12T14:34:49.778 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.819 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d84efd9e:::obj-1-435:head# 2020-09-12T14:34:49.820 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.861 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d883004b:::obj-1-162:head# 2020-09-12T14:34:49.861 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.903 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d8b60854:::obj-1-257:head# 2020-09-12T14:34:49.903 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.944 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d91aaae6:::obj-1-1:head# 2020-09-12T14:34:49.945 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:49.986 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d98695ca:::obj-1-524:head# 2020-09-12T14:34:49.986 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.028 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:d9ae97c8:::obj-1-177:head# 2020-09-12T14:34:50.028 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.069 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:da1713bd:::obj-1-309:head# 2020-09-12T14:34:50.070 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.111 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:da37824c:::obj-1-2:head# 2020-09-12T14:34:50.111 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.144 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:dab221b0:::obj-1-526:head# 2020-09-12T14:34:50.145 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.226 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:db27d885:::obj-1-363:head# 2020-09-12T14:34:50.227 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.261 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:dc001d76:::obj-1-329:head# 2020-09-12T14:34:50.262 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.294 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:dc4f4119:::obj-1-122:head# 2020-09-12T14:34:50.295 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.345 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:dcc36eb7:::obj-1-283:head# 2020-09-12T14:34:50.345 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.419 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:df7da06c:::obj-1-124:head# 2020-09-12T14:34:50.419 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.453 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:df7fcacb:::obj-1-330:head# 2020-09-12T14:34:50.453 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.486 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e05cbe31:::obj-1-413:head# 2020-09-12T14:34:50.487 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.519 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e1060a54:::obj-1-384:head# 2020-09-12T14:34:50.520 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.553 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e15b839b:::obj-1-316:head# 2020-09-12T14:34:50.553 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.586 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e161b6e5:::obj-1-103:head# 2020-09-12T14:34:50.586 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.645 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e1d1d79c:::obj-1-512:head# 2020-09-12T14:34:50.645 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.695 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e224b482:::obj-1-501:head# 2020-09-12T14:34:50.695 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.728 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e22d6f76:::obj-1-70:head# 2020-09-12T14:34:50.728 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.761 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e2bd801c:::obj-1-523:head# 2020-09-12T14:34:50.762 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.795 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e3d9f395:::obj-1-397:head# 2020-09-12T14:34:50.795 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.828 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e474ea25:::obj-1-457:head# 2020-09-12T14:34:50.828 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.861 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e49e1ff3:::obj-1-130:head# 2020-09-12T14:34:50.862 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.895 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e56c67a0:::obj-1-191:head# 2020-09-12T14:34:50.895 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.928 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e576e921:::obj-1-376:head# 2020-09-12T14:34:50.928 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.961 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e619fd26:::obj-1-395:head# 2020-09-12T14:34:50.962 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:50.995 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e625bdc5:::obj-1-522:head# 2020-09-12T14:34:50.995 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.028 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e68a0fa6:::obj-1-394:head# 2020-09-12T14:34:51.028 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.061 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e6cf260f:::obj-1-77:head# 2020-09-12T14:34:51.062 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.095 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e6d6255a:::obj-1-108:head# 2020-09-12T14:34:51.095 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.128 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e7a35d2f:::obj-1-116:head# 2020-09-12T14:34:51.128 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.153 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e7b07de5:::obj-1-238:head# 2020-09-12T14:34:51.154 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.187 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e8037aba:::obj-1-404:head# 2020-09-12T14:34:51.187 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.220 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e8e8b34f:::obj-1-291:head# 2020-09-12T14:34:51.220 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.253 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:e987cd7a:::obj-1-123:head# 2020-09-12T14:34:51.254 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.278 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:ea534db1:::obj-1-311:head# 2020-09-12T14:34:51.279 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.312 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:eabcc188:::obj-1-325:head# 2020-09-12T14:34:51.312 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.337 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:eb086660:::obj-1-595:head# 2020-09-12T14:34:51.337 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.362 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:eb2235d3:::obj-1-596:head# 2020-09-12T14:34:51.362 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.387 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:eb35ae40:::obj-1-516:head# 2020-09-12T14:34:51.387 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.412 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:eb360a13:::obj-1-224:head# 2020-09-12T14:34:51.412 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.437 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:eb3a3ea1:::obj-1-120:head# 2020-09-12T14:34:51.437 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.462 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:ebca9f6e:::obj-1-290:head# 2020-09-12T14:34:51.462 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.487 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:ec81db71:::obj-1-478:head# 2020-09-12T14:34:51.487 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.512 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:edeafa4c:::obj-1-362:head# 2020-09-12T14:34:51.512 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.537 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:eeb5261b:::obj-1-539:head# 2020-09-12T14:34:51.537 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.562 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:eec00eee:::obj-1-567:head# 2020-09-12T14:34:51.562 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.587 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:ef075e10:::obj-1-463:head# 2020-09-12T14:34:51.587 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.612 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:ef8e37d9:::obj-1-94:head# 2020-09-12T14:34:51.612 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.645 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:efad7f67:::obj-1-197:head# 2020-09-12T14:34:51.645 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.679 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:efe09ea7:::obj-1-537:head# 2020-09-12T14:34:51.679 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.712 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f0445019:::obj-1-480:head# 2020-09-12T14:34:51.712 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.754 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f08a432c:::obj-1-489:head# 2020-09-12T14:34:51.754 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.787 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f0bd9006:::obj-1-487:head# 2020-09-12T14:34:51.787 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.820 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f11a115d:::obj-1-591:head# 2020-09-12T14:34:51.821 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.853 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f232e0af:::obj-1-146:head# 2020-09-12T14:34:51.854 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.887 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f258db54:::obj-1-521:head# 2020-09-12T14:34:51.888 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.920 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f29cc204:::obj-1-566:head# 2020-09-12T14:34:51.921 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.954 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f2ae2451:::obj-1-66:head# 2020-09-12T14:34:51.954 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:51.987 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f318810a:::obj-1-160:head# 2020-09-12T14:34:51.987 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.020 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f3c35a0d:::obj-1-67:head# 2020-09-12T14:34:52.020 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.054 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f3ce9c90:::obj-1-88:head# 2020-09-12T14:34:52.054 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.087 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f3e3e81f:::obj-1-304:head# 2020-09-12T14:34:52.088 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.120 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f3f65b0a:::obj-1-367:head# 2020-09-12T14:34:52.121 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.154 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f4ea014b:::obj-1-149:head# 2020-09-12T14:34:52.154 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.187 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f52d115b:::obj-1-383:head# 2020-09-12T14:34:52.188 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.220 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f590311e:::obj-1-76:head# 2020-09-12T14:34:52.221 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.262 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f5a37734:::obj-1-51:head# 2020-09-12T14:34:52.263 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.296 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f5e2e77c:::obj-1-179:head# 2020-09-12T14:34:52.296 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.329 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f5e381bc:::obj-1-173:head# 2020-09-12T14:34:52.329 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.411 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f6c0ae0b:::obj-1-247:head# 2020-09-12T14:34:52.411 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.446 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f6db8a82:::obj-1-548:head# 2020-09-12T14:34:52.446 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.479 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f6e2dc46:::obj-1-36:head# 2020-09-12T14:34:52.479 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.512 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f76ee569:::obj-1-587:head# 2020-09-12T14:34:52.513 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.546 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f773b9ed:::obj-1-550:head# 2020-09-12T14:34:52.546 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.579 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f7e1adad:::obj-1-350:head# 2020-09-12T14:34:52.579 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.628 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f7eebf7a:::obj-1-559:head# 2020-09-12T14:34:52.629 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.687 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f877f025:::obj-1-187:head# 2020-09-12T14:34:52.688 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.721 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f992f9e9:::obj-1-402:head# 2020-09-12T14:34:52.721 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.754 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:f9fe960a:::obj-1-418:head# 2020-09-12T14:34:52.754 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.787 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:fa9b882f:::obj-1-165:head# 2020-09-12T14:34:52.788 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.821 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:fbd6461c:::obj-1-339:head# 2020-09-12T14:34:52.821 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.854 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:fbeab28d:::obj-1-431:head# 2020-09-12T14:34:52.854 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.887 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:fdea4138:::obj-1-185:head# 2020-09-12T14:34:52.888 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.912 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:fdfa7c9b:::obj-1-183:head# 2020-09-12T14:34:52.913 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.954 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:fe1ba57e:::obj-1-90:head# 2020-09-12T14:34:52.954 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:52.988 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:feb7adea:::obj-1-560:head# 2020-09-12T14:34:52.988 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:53.012 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:ffab306e:::obj-1-97:head# 2020-09-12T14:34:53.013 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:53.037 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:ffdfd397:::obj-1-433:head# 2020-09-12T14:34:53.038 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:53.063 INFO:tasks.workunit.client.0.smithi027.stdout:Write #2:ffe8cf04:::obj-1-303:head# 2020-09-12T14:34:53.063 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:53.088 INFO:tasks.workunit.client.0.smithi027.stdout:write_pg epoch 39 info 2.0( v 39'600 (39'500,39'600] local-lis/les=37/38 n=600 ec=26/26 lis/c=37/37 les/c/f=38/38/0 sis=37) 2020-09-12T14:34:53.121 INFO:tasks.workunit.client.0.smithi027.stdout:Import successful 2020-09-12T14:34:53.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:493: TEST_backfill_multi_partial: _objectstore_tool_nodown td/osd-backfill-space 1 --op import --pgid 1.0 --file td/osd-backfill-space/fillexport.out 2020-09-12T14:34:53.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1195: _objectstore_tool_nodown: local dir=td/osd-backfill-space 2020-09-12T14:34:53.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: _objectstore_tool_nodown: shift 2020-09-12T14:34:53.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: _objectstore_tool_nodown: local id=1 2020-09-12T14:34:53.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1198: _objectstore_tool_nodown: shift 2020-09-12T14:34:53.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: _objectstore_tool_nodown: local osd_data=td/osd-backfill-space/1 2020-09-12T14:34:53.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1202: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-backfill-space/1 --op import --pgid 1.0 --file td/osd-backfill-space/fillexport.out 2020-09-12T14:34:54.571 INFO:tasks.workunit.client.0.smithi027.stdout:get_pg_num_history pg_num_history pg_num_history(e43 pg_nums {1={21=1},2={26=1},3={31=1}} deleted_pools ) 2020-09-12T14:34:54.572 INFO:tasks.workunit.client.0.smithi027.stdout:Importing pgid 1.0 2020-09-12T14:34:54.572 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:0002d87b:::obj-fill-318:head# 2020-09-12T14:34:54.572 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:54.683 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:00676868:::obj-fill-583:head# 2020-09-12T14:34:54.684 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:54.721 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:00f4514e:::obj-fill-547:head# 2020-09-12T14:34:54.722 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:54.755 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:01596a3c:::obj-fill-533:head# 2020-09-12T14:34:54.755 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:54.788 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:0285e89c:::obj-fill-190:head# 2020-09-12T14:34:54.788 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:54.821 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:02bd591e:::obj-fill-463:head# 2020-09-12T14:34:54.822 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:54.855 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:02c01ffe:::obj-fill-28:head# 2020-09-12T14:34:54.855 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:54.888 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:03ada31e:::obj-fill-285:head# 2020-09-12T14:34:54.888 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:54.921 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:03b07c3f:::obj-fill-568:head# 2020-09-12T14:34:54.921 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:54.955 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:03b85770:::obj-fill-573:head# 2020-09-12T14:34:54.955 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:54.988 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:03dcbe89:::obj-fill-375:head# 2020-09-12T14:34:54.988 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.021 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:04231826:::obj-fill-594:head# 2020-09-12T14:34:55.022 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.055 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:04d48684:::obj-fill-421:head# 2020-09-12T14:34:55.055 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.088 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:05385fcc:::obj-fill-299:head# 2020-09-12T14:34:55.088 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.122 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:05785d38:::obj-fill-159:head# 2020-09-12T14:34:55.122 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.155 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:057ec231:::obj-fill-138:head# 2020-09-12T14:34:55.155 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.194 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:06058b5e:::obj-fill-569:head# 2020-09-12T14:34:55.194 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.260 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:06d2a3d9:::obj-fill-284:head# 2020-09-12T14:34:55.261 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.296 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:07597b63:::obj-fill-459:head# 2020-09-12T14:34:55.297 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.330 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:075f4069:::obj-fill-550:head# 2020-09-12T14:34:55.330 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.363 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:08234052:::obj-fill-17:head# 2020-09-12T14:34:55.363 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.396 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:091ce2ec:::obj-fill-472:head# 2020-09-12T14:34:55.397 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.430 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:095fd509:::obj-fill-210:head# 2020-09-12T14:34:55.430 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.463 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:0a52a594:::obj-fill-116:head# 2020-09-12T14:34:55.463 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.513 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:0a63716b:::obj-fill-225:head# 2020-09-12T14:34:55.513 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.547 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:0b0fa6fb:::obj-fill-541:head# 2020-09-12T14:34:55.547 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.580 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:0b673f72:::obj-fill-232:head# 2020-09-12T14:34:55.580 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.613 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:0bbcd1b2:::obj-fill-582:head# 2020-09-12T14:34:55.614 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.646 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:0caf8ce3:::obj-fill-214:head# 2020-09-12T14:34:55.647 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.680 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:0d94b9fb:::obj-fill-415:head# 2020-09-12T14:34:55.680 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.713 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:0de4e6f7:::obj-fill-94:head# 2020-09-12T14:34:55.714 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.747 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:0ef75768:::obj-fill-489:head# 2020-09-12T14:34:55.747 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.780 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:0f67284f:::obj-fill-213:head# 2020-09-12T14:34:55.780 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.813 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:0f823df2:::obj-fill-234:head# 2020-09-12T14:34:55.814 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.847 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:1005c08d:::obj-fill-490:head# 2020-09-12T14:34:55.847 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.880 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:1039525f:::obj-fill-369:head# 2020-09-12T14:34:55.880 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.913 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:105fb36c:::obj-fill-226:head# 2020-09-12T14:34:55.914 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.947 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:111abdeb:::obj-fill-554:head# 2020-09-12T14:34:55.947 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:55.980 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:11392f3d:::obj-fill-453:head# 2020-09-12T14:34:55.980 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.013 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:1169976c:::obj-fill-417:head# 2020-09-12T14:34:56.014 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.047 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:11840a2a:::obj-fill-477:head# 2020-09-12T14:34:56.047 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.080 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:11a4832e:::obj-fill-279:head# 2020-09-12T14:34:56.080 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.105 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:11a4d2f6:::obj-fill-256:head# 2020-09-12T14:34:56.105 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.130 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:12d52a31:::obj-fill-22:head# 2020-09-12T14:34:56.131 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.155 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:133c45db:::obj-fill-257:head# 2020-09-12T14:34:56.156 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.180 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:13563256:::obj-fill-150:head# 2020-09-12T14:34:56.180 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.205 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:13c09e00:::obj-fill-71:head# 2020-09-12T14:34:56.205 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.230 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:13e9fbed:::obj-fill-242:head# 2020-09-12T14:34:56.230 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.255 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:13fae6c2:::obj-fill-163:head# 2020-09-12T14:34:56.255 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.289 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:145025e0:::obj-fill-543:head# 2020-09-12T14:34:56.289 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.314 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:14517c84:::obj-fill-13:head# 2020-09-12T14:34:56.314 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.365 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:14802e9c:::obj-fill-127:head# 2020-09-12T14:34:56.365 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.422 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:150f1c96:::obj-fill-62:head# 2020-09-12T14:34:56.423 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.447 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:155e90a2:::obj-fill-537:head# 2020-09-12T14:34:56.447 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.472 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:1632c776:::obj-fill-262:head# 2020-09-12T14:34:56.472 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.497 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:163bde28:::obj-fill-523:head# 2020-09-12T14:34:56.498 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.523 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:169c4ad0:::obj-fill-85:head# 2020-09-12T14:34:56.523 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.547 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:16c4da1d:::obj-fill-436:head# 2020-09-12T14:34:56.547 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.581 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:17b28c32:::obj-fill-345:head# 2020-09-12T14:34:56.581 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.633 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:17b34255:::obj-fill-577:head# 2020-09-12T14:34:56.634 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.681 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:17d2541f:::obj-fill-426:head# 2020-09-12T14:34:56.681 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.706 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:17ea19f2:::obj-fill-542:head# 2020-09-12T14:34:56.706 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.731 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:189155ea:::obj-fill-496:head# 2020-09-12T14:34:56.731 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.756 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:191a6d81:::obj-fill-548:head# 2020-09-12T14:34:56.756 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.781 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:1a964993:::obj-fill-576:head# 2020-09-12T14:34:56.781 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.834 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:1aa623dc:::obj-fill-356:head# 2020-09-12T14:34:56.835 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.864 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:1abe5c38:::obj-fill-561:head# 2020-09-12T14:34:56.864 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.889 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:1bddc06b:::obj-fill-7:head# 2020-09-12T14:34:56.889 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.922 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:1be86b71:::obj-fill-295:head# 2020-09-12T14:34:56.923 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.956 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:1c499bca:::obj-fill-240:head# 2020-09-12T14:34:56.956 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:56.989 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:1c545e27:::obj-fill-509:head# 2020-09-12T14:34:56.989 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.022 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:1c63c8f4:::obj-fill-4:head# 2020-09-12T14:34:57.023 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.056 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:1d5a8f59:::obj-fill-377:head# 2020-09-12T14:34:57.056 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.089 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:1d85b5df:::obj-fill-201:head# 2020-09-12T14:34:57.090 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.122 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:1d9a0d5b:::obj-fill-581:head# 2020-09-12T14:34:57.123 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.156 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:1dcf5667:::obj-fill-488:head# 2020-09-12T14:34:57.156 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.189 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:1fa3090d:::obj-fill-458:head# 2020-09-12T14:34:57.190 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.239 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:1fe937f6:::obj-fill-337:head# 2020-09-12T14:34:57.240 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.273 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:1ff54542:::obj-fill-349:head# 2020-09-12T14:34:57.273 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.306 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:20d9e0ca:::obj-fill-367:head# 2020-09-12T14:34:57.306 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.339 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:21b8e3bb:::obj-fill-538:head# 2020-09-12T14:34:57.340 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.373 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:227183d4:::obj-fill-235:head# 2020-09-12T14:34:57.373 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.406 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:229e5e5d:::obj-fill-522:head# 2020-09-12T14:34:57.406 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.439 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:234f8861:::obj-fill-476:head# 2020-09-12T14:34:57.440 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.473 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:23ac2c9d:::obj-fill-471:head# 2020-09-12T14:34:57.473 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.506 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:25db1742:::obj-fill-200:head# 2020-09-12T14:34:57.506 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.540 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:264f8545:::obj-fill-230:head# 2020-09-12T14:34:57.540 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.573 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:267ba97f:::obj-fill-493:head# 2020-09-12T14:34:57.573 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.606 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:26b067e8:::obj-fill-208:head# 2020-09-12T14:34:57.606 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.640 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:26f521cb:::obj-fill-300:head# 2020-09-12T14:34:57.640 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.673 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:27e70a11:::obj-fill-368:head# 2020-09-12T14:34:57.673 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.706 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:29357d4a:::obj-fill-456:head# 2020-09-12T14:34:57.706 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.740 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:29f2567e:::obj-fill-414:head# 2020-09-12T14:34:57.740 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.773 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:2a10cbb6:::obj-fill-228:head# 2020-09-12T14:34:57.773 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.806 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:2a1519a6:::obj-fill-148:head# 2020-09-12T14:34:57.807 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.840 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:2a686ca6:::obj-fill-411:head# 2020-09-12T14:34:57.840 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.873 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:2ab3f384:::obj-fill-77:head# 2020-09-12T14:34:57.874 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.907 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:2ae6a1d0:::obj-fill-187:head# 2020-09-12T14:34:57.907 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.940 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:2b890959:::obj-fill-222:head# 2020-09-12T14:34:57.940 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:57.973 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:2bcc26c4:::obj-fill-114:head# 2020-09-12T14:34:57.973 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.006 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:2bd3a3a2:::obj-fill-281:head# 2020-09-12T14:34:58.007 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.040 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:2c1a21d7:::obj-fill-505:head# 2020-09-12T14:34:58.040 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.073 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:2c255347:::obj-fill-95:head# 2020-09-12T14:34:58.073 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.107 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:2c2a3f62:::obj-fill-131:head# 2020-09-12T14:34:58.107 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.140 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:2c7d92ce:::obj-fill-442:head# 2020-09-12T14:34:58.140 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.173 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:2d1ad117:::obj-fill-443:head# 2020-09-12T14:34:58.174 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.207 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:2d57b2f7:::obj-fill-192:head# 2020-09-12T14:34:58.207 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.240 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:2d7b53ea:::obj-fill-79:head# 2020-09-12T14:34:58.240 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.273 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:2db25266:::obj-fill-121:head# 2020-09-12T14:34:58.274 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.307 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:2e27853e:::obj-fill-473:head# 2020-09-12T14:34:58.307 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.365 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:2f12cb21:::obj-fill-386:head# 2020-09-12T14:34:58.366 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.423 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:2f957332:::obj-fill-422:head# 2020-09-12T14:34:58.424 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.448 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:2feae493:::obj-fill-301:head# 2020-09-12T14:34:58.449 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.474 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:302c95d2:::obj-fill-110:head# 2020-09-12T14:34:58.474 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.498 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:302d6b76:::obj-fill-374:head# 2020-09-12T14:34:58.499 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.523 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:30581f34:::obj-fill-313:head# 2020-09-12T14:34:58.524 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.557 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:307e86ac:::obj-fill-381:head# 2020-09-12T14:34:58.557 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.582 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:30ab77bf:::obj-fill-275:head# 2020-09-12T14:34:58.582 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.633 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:31127e6d:::obj-fill-250:head# 2020-09-12T14:34:58.634 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.682 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:32c93500:::obj-fill-155:head# 2020-09-12T14:34:58.682 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.707 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:339880b7:::obj-fill-227:head# 2020-09-12T14:34:58.707 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.732 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:34069e1e:::obj-fill-292:head# 2020-09-12T14:34:58.732 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.757 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:346dff31:::obj-fill-157:head# 2020-09-12T14:34:58.757 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.782 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:34ade0f1:::obj-fill-575:head# 2020-09-12T14:34:58.782 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.807 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:34bbae43:::obj-fill-90:head# 2020-09-12T14:34:58.807 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.832 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:34e97cf8:::obj-fill-334:head# 2020-09-12T14:34:58.832 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.857 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:35d7ccc2:::obj-fill-510:head# 2020-09-12T14:34:58.857 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.890 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:35f519b6:::obj-fill-513:head# 2020-09-12T14:34:58.890 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.923 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:364727e8:::obj-fill-130:head# 2020-09-12T14:34:58.924 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.965 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:364c0361:::obj-fill-586:head# 2020-09-12T14:34:58.965 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:58.990 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:368e7baf:::obj-fill-546:head# 2020-09-12T14:34:58.991 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.015 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:36ffb224:::obj-fill-82:head# 2020-09-12T14:34:59.016 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.040 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:371cb800:::obj-fill-330:head# 2020-09-12T14:34:59.040 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.065 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:37d5378d:::obj-fill-175:head# 2020-09-12T14:34:59.065 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.090 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:385593f3:::obj-fill-325:head# 2020-09-12T14:34:59.091 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.115 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:388d5073:::obj-fill-317:head# 2020-09-12T14:34:59.116 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.140 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:394af079:::obj-fill-59:head# 2020-09-12T14:34:59.141 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.174 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:3a41ad37:::obj-fill-409:head# 2020-09-12T14:34:59.174 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.207 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:3b3c63d2:::obj-fill-136:head# 2020-09-12T14:34:59.208 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.241 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:3bb50131:::obj-fill-512:head# 2020-09-12T14:34:59.241 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.274 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:3bb52bb5:::obj-fill-188:head# 2020-09-12T14:34:59.274 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.307 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:3c0683d1:::obj-fill-372:head# 2020-09-12T14:34:59.307 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.341 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:3c67d3f4:::obj-fill-249:head# 2020-09-12T14:34:59.341 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.374 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:3c6a4a4c:::obj-fill-495:head# 2020-09-12T14:34:59.374 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.407 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:3c7fc8c9:::obj-fill-324:head# 2020-09-12T14:34:59.407 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.441 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:3d940c50:::obj-fill-591:head# 2020-09-12T14:34:59.441 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.474 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:3ed5508e:::obj-fill-558:head# 2020-09-12T14:34:59.474 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.507 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:3f598316:::obj-fill-389:head# 2020-09-12T14:34:59.507 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.540 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:401cd0ce:::obj-fill-251:head# 2020-09-12T14:34:59.541 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.574 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:40fae006:::obj-fill-14:head# 2020-09-12T14:34:59.574 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.608 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:4120b10f:::obj-fill-165:head# 2020-09-12T14:34:59.608 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.641 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:41b382c2:::obj-fill-289:head# 2020-09-12T14:34:59.641 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.674 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:4277cdea:::obj-fill-319:head# 2020-09-12T14:34:59.674 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.716 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:42978c3c:::obj-fill-174:head# 2020-09-12T14:34:59.716 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.749 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:42b08dc2:::obj-fill-553:head# 2020-09-12T14:34:59.749 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.782 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:42ec4bf6:::obj-fill-209:head# 2020-09-12T14:34:59.783 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.816 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:430c129d:::obj-fill-448:head# 2020-09-12T14:34:59.816 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.849 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:4335d553:::obj-fill-600:head# 2020-09-12T14:34:59.849 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.882 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:44804f62:::obj-fill-532:head# 2020-09-12T14:34:59.883 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.916 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:44954d54:::obj-fill-2:head# 2020-09-12T14:34:59.916 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.949 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:4502f904:::obj-fill-429:head# 2020-09-12T14:34:59.950 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:34:59.982 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:4568ddcb:::obj-fill-382:head# 2020-09-12T14:34:59.983 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.016 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:45939974:::obj-fill-563:head# 2020-09-12T14:35:00.016 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.049 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:462a5bbd:::obj-fill-567:head# 2020-09-12T14:35:00.049 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.082 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:47773ae4:::obj-fill-307:head# 2020-09-12T14:35:00.083 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.116 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:4802b263:::obj-fill-387:head# 2020-09-12T14:35:00.116 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.149 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:483d938e:::obj-fill-128:head# 2020-09-12T14:35:00.149 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.176 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:48a7ad29:::obj-fill-147:head# 2020-09-12T14:35:00.177 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.199 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:493b61ce:::obj-fill-454:head# 2020-09-12T14:35:00.200 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.242 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:4afb70fe:::obj-fill-186:head# 2020-09-12T14:35:00.242 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.291 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:4bff08e3:::obj-fill-352:head# 2020-09-12T14:35:00.344 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.345 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:4c01eabe:::obj-fill-370:head# 2020-09-12T14:35:00.345 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.368 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:4c131560:::obj-fill-6:head# 2020-09-12T14:35:00.368 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.416 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:4c4c5e14:::obj-fill-350:head# 2020-09-12T14:35:00.417 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.441 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:4c80dc2c:::obj-fill-357:head# 2020-09-12T14:35:00.442 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.466 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:4c9dee9b:::obj-fill-540:head# 2020-09-12T14:35:00.467 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.491 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:4ce40c18:::obj-fill-280:head# 2020-09-12T14:35:00.491 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.516 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:4d854fc3:::obj-fill-585:head# 2020-09-12T14:35:00.517 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.541 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:4f615b53:::obj-fill-101:head# 2020-09-12T14:35:00.541 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.566 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5042b30f:::obj-fill-84:head# 2020-09-12T14:35:00.567 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.591 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:507dc08b:::obj-fill-336:head# 2020-09-12T14:35:00.591 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.699 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:512fcf5a:::obj-fill-169:head# 2020-09-12T14:35:00.700 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.749 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:51a0c8f8:::obj-fill-578:head# 2020-09-12T14:35:00.750 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.775 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:51fd05a2:::obj-fill-425:head# 2020-09-12T14:35:00.775 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.799 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:52400ebc:::obj-fill-81:head# 2020-09-12T14:35:00.800 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.824 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5288d40c:::obj-fill-16:head# 2020-09-12T14:35:00.825 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.850 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5295b0ee:::obj-fill-224:head# 2020-09-12T14:35:00.850 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.883 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:531e4c60:::obj-fill-103:head# 2020-09-12T14:35:00.883 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.916 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:54259f28:::obj-fill-584:head# 2020-09-12T14:35:00.917 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.950 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:542d619c:::obj-fill-528:head# 2020-09-12T14:35:00.950 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:00.983 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5503ca3e:::obj-fill-494:head# 2020-09-12T14:35:00.983 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.016 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5555c48b:::obj-fill-403:head# 2020-09-12T14:35:01.017 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.050 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:558d1ec7:::obj-fill-447:head# 2020-09-12T14:35:01.050 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.100 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:55aa2233:::obj-fill-21:head# 2020-09-12T14:35:01.100 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.133 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:55aaf8c4:::obj-fill-29:head# 2020-09-12T14:35:01.133 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.166 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:55e246e1:::obj-fill-65:head# 2020-09-12T14:35:01.167 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.193 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:563b14a8:::obj-fill-120:head# 2020-09-12T14:35:01.193 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.218 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:56e1df25:::obj-fill-305:head# 2020-09-12T14:35:01.218 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.243 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5702979c:::obj-fill-73:head# 2020-09-12T14:35:01.243 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.268 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:570722ba:::obj-fill-219:head# 2020-09-12T14:35:01.269 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.293 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:57f7a09b:::obj-fill-571:head# 2020-09-12T14:35:01.294 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.327 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:585309ba:::obj-fill-144:head# 2020-09-12T14:35:01.327 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.360 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:585c7c9c:::obj-fill-437:head# 2020-09-12T14:35:01.360 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.393 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:587c63db:::obj-fill-182:head# 2020-09-12T14:35:01.394 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.427 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:58f34d6b:::obj-fill-598:head# 2020-09-12T14:35:01.427 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.460 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:58f40088:::obj-fill-298:head# 2020-09-12T14:35:01.460 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.493 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:591e150c:::obj-fill-32:head# 2020-09-12T14:35:01.494 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.535 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:59971ddb:::obj-fill-559:head# 2020-09-12T14:35:01.535 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.568 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5adafca4:::obj-fill-56:head# 2020-09-12T14:35:01.569 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.602 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5b32b04f:::obj-fill-484:head# 2020-09-12T14:35:01.602 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.635 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5bdaeb16:::obj-fill-99:head# 2020-09-12T14:35:01.635 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.668 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5c1d6f4b:::obj-fill-133:head# 2020-09-12T14:35:01.669 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.702 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5c92b8f1:::obj-fill-255:head# 2020-09-12T14:35:01.702 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.735 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5ca4850f:::obj-fill-238:head# 2020-09-12T14:35:01.735 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.769 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5d928a5b:::obj-fill-395:head# 2020-09-12T14:35:01.769 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.802 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5dd49320:::obj-fill-423:head# 2020-09-12T14:35:01.802 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.835 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5e7bc368:::obj-fill-506:head# 2020-09-12T14:35:01.835 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.868 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5ebd5ee3:::obj-fill-322:head# 2020-09-12T14:35:01.869 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.902 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5edeedf1:::obj-fill-557:head# 2020-09-12T14:35:01.902 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.935 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:5facf8e2:::obj-fill-15:head# 2020-09-12T14:35:01.936 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:01.977 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:603a2c76:::obj-fill-439:head# 2020-09-12T14:35:01.978 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.019 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:60b30714:::obj-fill-53:head# 2020-09-12T14:35:02.019 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.052 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:612b4a8a:::obj-fill-412:head# 2020-09-12T14:35:02.052 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.094 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:61a111b6:::obj-fill-87:head# 2020-09-12T14:35:02.094 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.127 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:63a0e8a2:::obj-fill-266:head# 2020-09-12T14:35:02.127 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.160 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:63aa4d60:::obj-fill-68:head# 2020-09-12T14:35:02.161 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.193 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:63ac5b95:::obj-fill-405:head# 2020-09-12T14:35:02.194 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.227 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:6415a9e3:::obj-fill-400:head# 2020-09-12T14:35:02.227 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.260 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:64ba03ae:::obj-fill-20:head# 2020-09-12T14:35:02.260 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.294 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:6512f015:::obj-fill-597:head# 2020-09-12T14:35:02.294 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.327 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:65433104:::obj-fill-504:head# 2020-09-12T14:35:02.327 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.410 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:65950c58:::obj-fill-86:head# 2020-09-12T14:35:02.410 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.444 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:65b6b2d7:::obj-fill-39:head# 2020-09-12T14:35:02.444 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.469 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:661c8afd:::obj-fill-267:head# 2020-09-12T14:35:02.469 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.494 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:66f87c94:::obj-fill-229:head# 2020-09-12T14:35:02.494 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.519 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:67292cef:::obj-fill-93:head# 2020-09-12T14:35:02.519 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.544 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:67f0e12c:::obj-fill-363:head# 2020-09-12T14:35:02.544 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.569 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:696eb83b:::obj-fill-185:head# 2020-09-12T14:35:02.569 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.594 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:6b33981c:::obj-fill-259:head# 2020-09-12T14:35:02.594 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.669 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:6b5a527c:::obj-fill-485:head# 2020-09-12T14:35:02.670 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.702 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:6b6f1852:::obj-fill-390:head# 2020-09-12T14:35:02.703 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.727 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:6ba5c5f8:::obj-fill-304:head# 2020-09-12T14:35:02.728 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.752 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:6bce929d:::obj-fill-518:head# 2020-09-12T14:35:02.753 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.777 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:6c8ffc04:::obj-fill-61:head# 2020-09-12T14:35:02.778 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.802 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:6ca7e551:::obj-fill-40:head# 2020-09-12T14:35:02.803 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.827 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:6caff2cd:::obj-fill-332:head# 2020-09-12T14:35:02.828 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.861 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:6ce4adc7:::obj-fill-574:head# 2020-09-12T14:35:02.861 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.886 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:6d09d770:::obj-fill-74:head# 2020-09-12T14:35:02.886 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.911 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:6e039822:::obj-fill-278:head# 2020-09-12T14:35:02.911 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.936 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:6f2cd109:::obj-fill-297:head# 2020-09-12T14:35:02.936 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.961 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:6f3754c4:::obj-fill-88:head# 2020-09-12T14:35:02.961 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:02.986 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:6f86cfd2:::obj-fill-511:head# 2020-09-12T14:35:02.986 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.011 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:6fe3f3f7:::obj-fill-193:head# 2020-09-12T14:35:03.011 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.036 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7002c57d:::obj-fill-50:head# 2020-09-12T14:35:03.036 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.061 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7030b2a3:::obj-fill-217:head# 2020-09-12T14:35:03.061 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.086 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:70665087:::obj-fill-51:head# 2020-09-12T14:35:03.086 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.111 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:70a0f5e0:::obj-fill-11:head# 2020-09-12T14:35:03.111 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.153 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:71f4643d:::obj-fill-236:head# 2020-09-12T14:35:03.153 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.178 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:722fbd0f:::obj-fill-302:head# 2020-09-12T14:35:03.178 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.203 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:72584da9:::obj-fill-445:head# 2020-09-12T14:35:03.203 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.228 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:72d91d24:::obj-fill-218:head# 2020-09-12T14:35:03.228 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.253 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:72f7268b:::obj-fill-106:head# 2020-09-12T14:35:03.253 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.278 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:74077abf:::obj-fill-435:head# 2020-09-12T14:35:03.278 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.303 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:740ca050:::obj-fill-112:head# 2020-09-12T14:35:03.303 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.336 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7435e881:::obj-fill-78:head# 2020-09-12T14:35:03.336 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.361 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:74ba0bb7:::obj-fill-478:head# 2020-09-12T14:35:03.361 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.386 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:74d82091:::obj-fill-96:head# 2020-09-12T14:35:03.386 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.411 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7515f56c:::obj-fill-483:head# 2020-09-12T14:35:03.412 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.436 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7550d6fe:::obj-fill-286:head# 2020-09-12T14:35:03.436 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.461 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:75807edc:::obj-fill-223:head# 2020-09-12T14:35:03.462 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.486 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:762ba324:::obj-fill-172:head# 2020-09-12T14:35:03.486 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.520 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:76369bda:::obj-fill-570:head# 2020-09-12T14:35:03.520 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.553 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7667a5b0:::obj-fill-196:head# 2020-09-12T14:35:03.553 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.586 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:77736c0c:::obj-fill-566:head# 2020-09-12T14:35:03.587 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.620 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:77fb47a4:::obj-fill-314:head# 2020-09-12T14:35:03.620 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.653 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:785ad894:::obj-fill-80:head# 2020-09-12T14:35:03.653 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.695 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:785e3d14:::obj-fill-530:head# 2020-09-12T14:35:03.695 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.728 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:788efef4:::obj-fill-595:head# 2020-09-12T14:35:03.728 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.778 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:789ec5ab:::obj-fill-44:head# 2020-09-12T14:35:03.778 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.811 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:798a3fa8:::obj-fill-517:head# 2020-09-12T14:35:03.812 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.845 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:79a105ef:::obj-fill-315:head# 2020-09-12T14:35:03.845 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.878 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:79d02f04:::obj-fill-243:head# 2020-09-12T14:35:03.878 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.911 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7a00dfcb:::obj-fill-402:head# 2020-09-12T14:35:03.912 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.945 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7b982684:::obj-fill-168:head# 2020-09-12T14:35:03.945 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:03.978 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7bb36851:::obj-fill-440:head# 2020-09-12T14:35:03.978 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.011 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7bb49d8c:::obj-fill-55:head# 2020-09-12T14:35:04.012 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.045 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7c77b783:::obj-fill-580:head# 2020-09-12T14:35:04.045 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.087 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7c89d75b:::obj-fill-555:head# 2020-09-12T14:35:04.087 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.120 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7d12fb67:::obj-fill-399:head# 2020-09-12T14:35:04.120 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.153 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7d139961:::obj-fill-92:head# 2020-09-12T14:35:04.153 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.187 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7d39ffd0:::obj-fill-135:head# 2020-09-12T14:35:04.187 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.220 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7e534cc4:::obj-fill-282:head# 2020-09-12T14:35:04.220 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.253 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7e7a08e0:::obj-fill-108:head# 2020-09-12T14:35:04.254 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.287 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7e8e2b67:::obj-fill-244:head# 2020-09-12T14:35:04.287 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.320 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7eea215e:::obj-fill-139:head# 2020-09-12T14:35:04.320 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.369 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7efa3c6e:::obj-fill-408:head# 2020-09-12T14:35:04.369 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.470 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7f03a24b:::obj-fill-45:head# 2020-09-12T14:35:04.470 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.503 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7f6a2e2b:::obj-fill-427:head# 2020-09-12T14:35:04.504 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.537 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7fb0abd6:::obj-fill-479:head# 2020-09-12T14:35:04.537 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.570 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:7fb77de1:::obj-fill-167:head# 2020-09-12T14:35:04.570 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.612 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:80012a03:::obj-fill-105:head# 2020-09-12T14:35:04.612 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.670 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:80075ae5:::obj-fill-376:head# 2020-09-12T14:35:04.670 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.695 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8089baa5:::obj-fill-97:head# 2020-09-12T14:35:04.696 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.720 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:80d0b4c5:::obj-fill-123:head# 2020-09-12T14:35:04.721 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.746 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:810f319f:::obj-fill-552:head# 2020-09-12T14:35:04.746 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.770 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:81427ef2:::obj-fill-474:head# 2020-09-12T14:35:04.771 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.795 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:818d780c:::obj-fill-33:head# 2020-09-12T14:35:04.795 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.820 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:819f31b9:::obj-fill-366:head# 2020-09-12T14:35:04.820 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.845 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8205549d:::obj-fill-125:head# 2020-09-12T14:35:04.846 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.870 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:821f56e5:::obj-fill-339:head# 2020-09-12T14:35:04.871 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.895 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:822807c6:::obj-fill-69:head# 2020-09-12T14:35:04.896 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.920 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:822fcf61:::obj-fill-560:head# 2020-09-12T14:35:04.921 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.945 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:82ad696b:::obj-fill-149:head# 2020-09-12T14:35:04.946 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.970 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:835cc771:::obj-fill-446:head# 2020-09-12T14:35:04.971 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:04.995 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8386bee1:::obj-fill-137:head# 2020-09-12T14:35:04.996 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.020 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:83c1303a:::obj-fill-311:head# 2020-09-12T14:35:05.021 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.045 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:83cf1c2a:::obj-fill-360:head# 2020-09-12T14:35:05.046 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.071 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:83d70321:::obj-fill-351:head# 2020-09-12T14:35:05.071 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.095 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:83ef162d:::obj-fill-331:head# 2020-09-12T14:35:05.096 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.120 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:840f8f20:::obj-fill-398:head# 2020-09-12T14:35:05.121 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.145 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:84d2e456:::obj-fill-207:head# 2020-09-12T14:35:05.146 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.187 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:856697ff:::obj-fill-333:head# 2020-09-12T14:35:05.187 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.212 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:858be937:::obj-fill-418:head# 2020-09-12T14:35:05.212 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.237 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:85a0b302:::obj-fill-438:head# 2020-09-12T14:35:05.238 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.281 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:85fa942e:::obj-fill-272:head# 2020-09-12T14:35:05.281 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.329 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:863eb6f8:::obj-fill-320:head# 2020-09-12T14:35:05.329 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.354 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:868d4588:::obj-fill-19:head# 2020-09-12T14:35:05.354 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.379 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8759d023:::obj-fill-466:head# 2020-09-12T14:35:05.379 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.404 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:88072111:::obj-fill-312:head# 2020-09-12T14:35:05.404 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.429 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:886edcef:::obj-fill-380:head# 2020-09-12T14:35:05.429 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.454 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:88ce753e:::obj-fill-43:head# 2020-09-12T14:35:05.454 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.479 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:890e1d83:::obj-fill-590:head# 2020-09-12T14:35:05.479 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.504 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:891566a3:::obj-fill-341:head# 2020-09-12T14:35:05.504 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.529 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:891ca235:::obj-fill-177:head# 2020-09-12T14:35:05.529 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.554 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:897fa9d0:::obj-fill-283:head# 2020-09-12T14:35:05.554 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.579 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:89cd304b:::obj-fill-592:head# 2020-09-12T14:35:05.579 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.604 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:89d04c4a:::obj-fill-388:head# 2020-09-12T14:35:05.604 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.629 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:89db45d9:::obj-fill-247:head# 2020-09-12T14:35:05.629 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.654 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8ad879c5:::obj-fill-492:head# 2020-09-12T14:35:05.654 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.687 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8af4b5de:::obj-fill-37:head# 2020-09-12T14:35:05.688 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.721 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8b132e86:::obj-fill-35:head# 2020-09-12T14:35:05.721 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.754 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8b80a60a:::obj-fill-100:head# 2020-09-12T14:35:05.754 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.788 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8bb64fa6:::obj-fill-599:head# 2020-09-12T14:35:05.788 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.821 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8bed03f5:::obj-fill-5:head# 2020-09-12T14:35:05.821 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.854 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8c60e135:::obj-fill-470:head# 2020-09-12T14:35:05.854 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.887 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8c774623:::obj-fill-38:head# 2020-09-12T14:35:05.888 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.921 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8cc528ae:::obj-fill-519:head# 2020-09-12T14:35:05.921 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.954 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8cf73826:::obj-fill-460:head# 2020-09-12T14:35:05.954 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:05.996 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8d3564b3:::obj-fill-452:head# 2020-09-12T14:35:05.996 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.029 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8dfe1ca2:::obj-fill-469:head# 2020-09-12T14:35:06.029 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.063 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8e527672:::obj-fill-246:head# 2020-09-12T14:35:06.063 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.096 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8e96103f:::obj-fill-444:head# 2020-09-12T14:35:06.096 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.129 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8ed6417f:::obj-fill-146:head# 2020-09-12T14:35:06.130 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.163 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8eda2ffc:::obj-fill-539:head# 2020-09-12T14:35:06.163 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.196 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8f08dd48:::obj-fill-26:head# 2020-09-12T14:35:06.196 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.229 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8f111b25:::obj-fill-18:head# 2020-09-12T14:35:06.230 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.263 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:8fde63f8:::obj-fill-338:head# 2020-09-12T14:35:06.263 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.296 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:9030746d:::obj-fill-468:head# 2020-09-12T14:35:06.296 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.329 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:9115d8a1:::obj-fill-260:head# 2020-09-12T14:35:06.329 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.411 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:91649909:::obj-fill-140:head# 2020-09-12T14:35:06.411 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.446 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:91c1081d:::obj-fill-404:head# 2020-09-12T14:35:06.446 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.479 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:91f42503:::obj-fill-216:head# 2020-09-12T14:35:06.480 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.513 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:927b34a0:::obj-fill-63:head# 2020-09-12T14:35:06.513 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.546 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:92a00a91:::obj-fill-467:head# 2020-09-12T14:35:06.546 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.579 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:930a7268:::obj-fill-450:head# 2020-09-12T14:35:06.580 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.628 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:93c3d0bf:::obj-fill-118:head# 2020-09-12T14:35:06.629 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.688 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:93ca7c9e:::obj-fill-132:head# 2020-09-12T14:35:06.688 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.723 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:942435d5:::obj-fill-378:head# 2020-09-12T14:35:06.724 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.754 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:9445f8c5:::obj-fill-245:head# 2020-09-12T14:35:06.755 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.779 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:9457c8d9:::obj-fill-535:head# 2020-09-12T14:35:06.780 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.804 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:950dc56e:::obj-fill-348:head# 2020-09-12T14:35:06.805 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.829 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:9559ee1d:::obj-fill-545:head# 2020-09-12T14:35:06.830 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.854 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:957c5cc4:::obj-fill-212:head# 2020-09-12T14:35:06.855 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.879 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:95bdac02:::obj-fill-588:head# 2020-09-12T14:35:06.880 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.905 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:95f202af:::obj-fill-126:head# 2020-09-12T14:35:06.905 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.930 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:96789003:::obj-fill-83:head# 2020-09-12T14:35:06.930 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:06.955 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:96d494a8:::obj-fill-316:head# 2020-09-12T14:35:06.955 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.005 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:97adb85b:::obj-fill-233:head# 2020-09-12T14:35:07.005 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.030 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:980d998c:::obj-fill-197:head# 2020-09-12T14:35:07.030 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.055 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:98235c44:::obj-fill-396:head# 2020-09-12T14:35:07.055 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.080 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:987d3b87:::obj-fill-327:head# 2020-09-12T14:35:07.080 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.105 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:990a5e24:::obj-fill-515:head# 2020-09-12T14:35:07.105 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.130 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:995fec05:::obj-fill-191:head# 2020-09-12T14:35:07.130 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.155 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:9a769b75:::obj-fill-587:head# 2020-09-12T14:35:07.155 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.180 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:9b141990:::obj-fill-394:head# 2020-09-12T14:35:07.180 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.213 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:9bb108fb:::obj-fill-579:head# 2020-09-12T14:35:07.213 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.238 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:9ce19d92:::obj-fill-361:head# 2020-09-12T14:35:07.238 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.263 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:9d8c3f95:::obj-fill-572:head# 2020-09-12T14:35:07.263 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.288 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:9e283e00:::obj-fill-254:head# 2020-09-12T14:35:07.288 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.313 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:9e3a1a61:::obj-fill-58:head# 2020-09-12T14:35:07.313 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.338 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:9e8f6029:::obj-fill-41:head# 2020-09-12T14:35:07.338 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.363 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:9ef117aa:::obj-fill-520:head# 2020-09-12T14:35:07.363 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.388 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:9f3bdfc2:::obj-fill-184:head# 2020-09-12T14:35:07.388 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.413 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:9fd89320:::obj-fill-364:head# 2020-09-12T14:35:07.413 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.438 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:9feb7062:::obj-fill-419:head# 2020-09-12T14:35:07.439 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.463 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a07504c4:::obj-fill-516:head# 2020-09-12T14:35:07.464 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.488 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a1897e34:::obj-fill-153:head# 2020-09-12T14:35:07.489 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.513 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a19a9d76:::obj-fill-30:head# 2020-09-12T14:35:07.514 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.538 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a1a49c98:::obj-fill-98:head# 2020-09-12T14:35:07.539 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.563 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a1ea911a:::obj-fill-36:head# 2020-09-12T14:35:07.564 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.588 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a24bad98:::obj-fill-335:head# 2020-09-12T14:35:07.589 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.613 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a263b16f:::obj-fill-326:head# 2020-09-12T14:35:07.614 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.638 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a27d661e:::obj-fill-152:head# 2020-09-12T14:35:07.639 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.663 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a282a129:::obj-fill-451:head# 2020-09-12T14:35:07.664 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.688 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a28f05ba:::obj-fill-241:head# 2020-09-12T14:35:07.689 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.713 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a2cdc5cc:::obj-fill-180:head# 2020-09-12T14:35:07.714 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.738 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a30777ea:::obj-fill-486:head# 2020-09-12T14:35:07.739 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.772 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a3a65661:::obj-fill-151:head# 2020-09-12T14:35:07.772 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.813 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a3cb2967:::obj-fill-195:head# 2020-09-12T14:35:07.814 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.847 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a3ffc098:::obj-fill-293:head# 2020-09-12T14:35:07.847 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.880 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a4a31ccc:::obj-fill-413:head# 2020-09-12T14:35:07.880 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.913 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a51c3677:::obj-fill-384:head# 2020-09-12T14:35:07.914 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.947 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a5e941a7:::obj-fill-141:head# 2020-09-12T14:35:07.947 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:07.980 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a6cdd8b2:::obj-fill-342:head# 2020-09-12T14:35:07.981 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.014 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a6e20b86:::obj-fill-593:head# 2020-09-12T14:35:08.014 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.055 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a7a9ccf9:::obj-fill-129:head# 2020-09-12T14:35:08.055 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.097 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a7b702d3:::obj-fill-49:head# 2020-09-12T14:35:08.097 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.139 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a8daf8bd:::obj-fill-391:head# 2020-09-12T14:35:08.139 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.180 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:a99c3cef:::obj-fill-171:head# 2020-09-12T14:35:08.181 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.222 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:aabb5718:::obj-fill-521:head# 2020-09-12T14:35:08.222 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.255 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ab0cb851:::obj-fill-183:head# 2020-09-12T14:35:08.256 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.289 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ab7a05e4:::obj-fill-203:head# 2020-09-12T14:35:08.289 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.322 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ab7dda83:::obj-fill-113:head# 2020-09-12T14:35:08.322 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.370 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:abb3502c:::obj-fill-461:head# 2020-09-12T14:35:08.370 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.430 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ac1c41be:::obj-fill-162:head# 2020-09-12T14:35:08.431 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.464 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ac2114ba:::obj-fill-239:head# 2020-09-12T14:35:08.464 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.497 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ad355fca:::obj-fill-271:head# 2020-09-12T14:35:08.498 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.530 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ad4634a0:::obj-fill-487:head# 2020-09-12T14:35:08.531 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.564 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:add2bb13:::obj-fill-31:head# 2020-09-12T14:35:08.564 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.597 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:aeb6258d:::obj-fill-124:head# 2020-09-12T14:35:08.598 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.679 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:af17f7d7:::obj-fill-464:head# 2020-09-12T14:35:08.680 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.714 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:af1e1cf0:::obj-fill-237:head# 2020-09-12T14:35:08.715 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.747 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:afc439ce:::obj-fill-248:head# 2020-09-12T14:35:08.748 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.781 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:afceab4d:::obj-fill-527:head# 2020-09-12T14:35:08.781 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.814 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b03af40c:::obj-fill-143:head# 2020-09-12T14:35:08.814 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.847 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b04cf0db:::obj-fill-507:head# 2020-09-12T14:35:08.848 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.889 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b07896f0:::obj-fill-269:head# 2020-09-12T14:35:08.889 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.922 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b0b954d5:::obj-fill-524:head# 2020-09-12T14:35:08.923 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.956 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b1cbfcb7:::obj-fill-134:head# 2020-09-12T14:35:08.956 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:08.989 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b2978fd3:::obj-fill-27:head# 2020-09-12T14:35:08.989 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.022 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b394798f:::obj-fill-156:head# 2020-09-12T14:35:09.023 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.056 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b3c6f36d:::obj-fill-401:head# 2020-09-12T14:35:09.056 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.089 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b3f8e5ac:::obj-fill-290:head# 2020-09-12T14:35:09.089 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.122 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b41ed5f8:::obj-fill-76:head# 2020-09-12T14:35:09.123 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.156 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b4878e76:::obj-fill-253:head# 2020-09-12T14:35:09.156 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.189 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b4e19741:::obj-fill-503:head# 2020-09-12T14:35:09.189 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.222 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b53a3690:::obj-fill-264:head# 2020-09-12T14:35:09.223 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.256 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b545ab4f:::obj-fill-24:head# 2020-09-12T14:35:09.256 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.289 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b55af5e6:::obj-fill-491:head# 2020-09-12T14:35:09.290 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.331 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b57edf82:::obj-fill-526:head# 2020-09-12T14:35:09.331 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.364 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b589d2be:::obj-fill-323:head# 2020-09-12T14:35:09.364 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.398 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b6207f32:::obj-fill-344:head# 2020-09-12T14:35:09.398 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.431 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b757ee2e:::obj-fill-433:head# 2020-09-12T14:35:09.431 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.464 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b75cc479:::obj-fill-34:head# 2020-09-12T14:35:09.464 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.498 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b7b03117:::obj-fill-42:head# 2020-09-12T14:35:09.498 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.531 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b7e6205d:::obj-fill-501:head# 2020-09-12T14:35:09.531 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.564 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b8d18b7a:::obj-fill-353:head# 2020-09-12T14:35:09.564 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.598 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b9189c29:::obj-fill-328:head# 2020-09-12T14:35:09.598 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.631 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b98e07aa:::obj-fill-54:head# 2020-09-12T14:35:09.631 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.664 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:b9cd1e9a:::obj-fill-371:head# 2020-09-12T14:35:09.665 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.698 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:bc3322bb:::obj-fill-340:head# 2020-09-12T14:35:09.698 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.731 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:bcf43161:::obj-fill-294:head# 2020-09-12T14:35:09.731 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.764 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:bd07fc4f:::obj-fill-549:head# 2020-09-12T14:35:09.765 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.798 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:bdf48e47:::obj-fill-430:head# 2020-09-12T14:35:09.798 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.831 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:be01c111:::obj-fill-1:head# 2020-09-12T14:35:09.831 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.864 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:be3a4bc7:::obj-fill-102:head# 2020-09-12T14:35:09.865 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.914 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:be8e1fcd:::obj-fill-321:head# 2020-09-12T14:35:09.916 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.956 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:bf5fcbac:::obj-fill-287:head# 2020-09-12T14:35:09.956 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:09.998 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:bf806d14:::obj-fill-115:head# 2020-09-12T14:35:09.998 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.040 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:bfb28823:::obj-fill-72:head# 2020-09-12T14:35:10.040 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.081 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:bfef805e:::obj-fill-119:head# 2020-09-12T14:35:10.082 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.123 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c02f8e10:::obj-fill-497:head# 2020-09-12T14:35:10.123 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.165 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c050cd6e:::obj-fill-8:head# 2020-09-12T14:35:10.165 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.215 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c218e297:::obj-fill-173:head# 2020-09-12T14:35:10.215 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.256 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c2bc66fe:::obj-fill-252:head# 2020-09-12T14:35:10.257 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.298 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c3b55903:::obj-fill-428:head# 2020-09-12T14:35:10.298 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.370 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c51907b0:::obj-fill-457:head# 2020-09-12T14:35:10.370 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.420 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c561dc83:::obj-fill-75:head# 2020-09-12T14:35:10.420 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.481 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c5d1e1a9:::obj-fill-268:head# 2020-09-12T14:35:10.482 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.523 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c60ab0d0:::obj-fill-303:head# 2020-09-12T14:35:10.523 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.557 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c644ce52:::obj-fill-514:head# 2020-09-12T14:35:10.557 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.629 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c6ab3a11:::obj-fill-407:head# 2020-09-12T14:35:10.630 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.665 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c74377bf:::obj-fill-432:head# 2020-09-12T14:35:10.665 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.698 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c767fff6:::obj-fill-531:head# 2020-09-12T14:35:10.698 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.731 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c7a9adad:::obj-fill-475:head# 2020-09-12T14:35:10.732 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.765 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c7e3a9c2:::obj-fill-465:head# 2020-09-12T14:35:10.765 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.798 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c850aed7:::obj-fill-424:head# 2020-09-12T14:35:10.799 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.831 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c8a0088a:::obj-fill-329:head# 2020-09-12T14:35:10.832 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.882 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:c9d419f9:::obj-fill-176:head# 2020-09-12T14:35:10.882 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.915 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:caf25ec2:::obj-fill-274:head# 2020-09-12T14:35:10.915 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.948 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:cb625830:::obj-fill-70:head# 2020-09-12T14:35:10.949 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:10.982 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:cbb65125:::obj-fill-529:head# 2020-09-12T14:35:10.982 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.015 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:cbe2544c:::obj-fill-3:head# 2020-09-12T14:35:11.015 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.048 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:cbfce4ee:::obj-fill-455:head# 2020-09-12T14:35:11.049 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.082 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:cc62419c:::obj-fill-498:head# 2020-09-12T14:35:11.082 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.115 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:cce831fa:::obj-fill-160:head# 2020-09-12T14:35:11.115 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.148 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:cda71af4:::obj-fill-122:head# 2020-09-12T14:35:11.149 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.182 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:cea7a728:::obj-fill-265:head# 2020-09-12T14:35:11.182 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.215 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:cee19f0a:::obj-fill-142:head# 2020-09-12T14:35:11.215 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.248 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:cf22117e:::obj-fill-499:head# 2020-09-12T14:35:11.249 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.282 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:cfbfa66d:::obj-fill-66:head# 2020-09-12T14:35:11.282 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.315 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:cfc2552c:::obj-fill-211:head# 2020-09-12T14:35:11.315 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.348 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:cfc563e8:::obj-fill-392:head# 2020-09-12T14:35:11.349 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.382 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d02d8880:::obj-fill-362:head# 2020-09-12T14:35:11.382 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.415 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d054947e:::obj-fill-67:head# 2020-09-12T14:35:11.415 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.448 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d061c977:::obj-fill-164:head# 2020-09-12T14:35:11.449 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.482 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d06a5f57:::obj-fill-296:head# 2020-09-12T14:35:11.482 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.515 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d110062a:::obj-fill-343:head# 2020-09-12T14:35:11.515 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.548 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d1b8ad47:::obj-fill-383:head# 2020-09-12T14:35:11.549 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.582 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d1e80db6:::obj-fill-117:head# 2020-09-12T14:35:11.582 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.615 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d210027e:::obj-fill-231:head# 2020-09-12T14:35:11.616 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.649 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d24428ed:::obj-fill-199:head# 2020-09-12T14:35:11.649 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.682 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d27c3c4b:::obj-fill-154:head# 2020-09-12T14:35:11.682 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.724 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d28957e3:::obj-fill-263:head# 2020-09-12T14:35:11.724 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.757 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d2d44da7:::obj-fill-544:head# 2020-09-12T14:35:11.757 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.790 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d31004a4:::obj-fill-434:head# 2020-09-12T14:35:11.791 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.824 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d32487c3:::obj-fill-385:head# 2020-09-12T14:35:11.824 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.857 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d340891c:::obj-fill-12:head# 2020-09-12T14:35:11.857 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.890 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d4135728:::obj-fill-310:head# 2020-09-12T14:35:11.891 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.924 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d5a43972:::obj-fill-9:head# 2020-09-12T14:35:11.924 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.957 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d5efeb15:::obj-fill-347:head# 2020-09-12T14:35:11.957 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:11.990 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d6463f70:::obj-fill-536:head# 2020-09-12T14:35:11.991 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.024 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d666d9b7:::obj-fill-420:head# 2020-09-12T14:35:12.024 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.057 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d6e86688:::obj-fill-166:head# 2020-09-12T14:35:12.057 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.099 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d730370d:::obj-fill-206:head# 2020-09-12T14:35:12.099 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.132 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d783551c:::obj-fill-109:head# 2020-09-12T14:35:12.132 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.166 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d7f77e5f:::obj-fill-551:head# 2020-09-12T14:35:12.166 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.199 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d843056c:::obj-fill-291:head# 2020-09-12T14:35:12.199 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.232 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d84eb173:::obj-fill-221:head# 2020-09-12T14:35:12.232 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.265 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d8dd069c:::obj-fill-60:head# 2020-09-12T14:35:12.266 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.307 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d8f2f750:::obj-fill-562:head# 2020-09-12T14:35:12.307 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.361 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d8f80ac2:::obj-fill-179:head# 2020-09-12T14:35:12.361 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.416 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:d9f904cf:::obj-fill-308:head# 2020-09-12T14:35:12.416 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.457 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:db0a534b:::obj-fill-410:head# 2020-09-12T14:35:12.458 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.499 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:dc4403f4:::obj-fill-416:head# 2020-09-12T14:35:12.499 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.541 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:dc66c3ca:::obj-fill-220:head# 2020-09-12T14:35:12.541 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.582 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:dc8198dc:::obj-fill-596:head# 2020-09-12T14:35:12.583 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.646 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:dcbade2b:::obj-fill-10:head# 2020-09-12T14:35:12.646 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.707 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ddb787ea:::obj-fill-556:head# 2020-09-12T14:35:12.708 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.749 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:de58065b:::obj-fill-306:head# 2020-09-12T14:35:12.749 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.791 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:deaa6fab:::obj-fill-194:head# 2020-09-12T14:35:12.791 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.832 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:dfee7714:::obj-fill-202:head# 2020-09-12T14:35:12.833 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.874 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:dff5df4a:::obj-fill-565:head# 2020-09-12T14:35:12.874 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.916 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:e2053442:::obj-fill-107:head# 2020-09-12T14:35:12.916 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.957 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:e32055b8:::obj-fill-57:head# 2020-09-12T14:35:12.958 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:12.991 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:e37fac1e:::obj-fill-161:head# 2020-09-12T14:35:12.991 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.024 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:e388f5aa:::obj-fill-273:head# 2020-09-12T14:35:13.024 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.057 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:e38b1f4d:::obj-fill-393:head# 2020-09-12T14:35:13.058 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.091 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:e4792474:::obj-fill-158:head# 2020-09-12T14:35:13.091 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.124 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:e4899cb0:::obj-fill-270:head# 2020-09-12T14:35:13.124 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.157 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:e62ae9f1:::obj-fill-288:head# 2020-09-12T14:35:13.158 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.191 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:e64fd95a:::obj-fill-449:head# 2020-09-12T14:35:13.191 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.224 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:e6682cfc:::obj-fill-373:head# 2020-09-12T14:35:13.225 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.257 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:e716717b:::obj-fill-178:head# 2020-09-12T14:35:13.258 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.291 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:e80226af:::obj-fill-354:head# 2020-09-12T14:35:13.291 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.324 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:e83a5940:::obj-fill-564:head# 2020-09-12T14:35:13.324 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.358 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:e854bfbe:::obj-fill-589:head# 2020-09-12T14:35:13.358 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.391 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:e8b197bc:::obj-fill-89:head# 2020-09-12T14:35:13.391 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.424 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:e92fca88:::obj-fill-397:head# 2020-09-12T14:35:13.425 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.458 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:e94d8f7b:::obj-fill-261:head# 2020-09-12T14:35:13.458 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.491 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:e98cc0a4:::obj-fill-525:head# 2020-09-12T14:35:13.491 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.524 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ea5a9459:::obj-fill-431:head# 2020-09-12T14:35:13.525 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.558 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ebc54d47:::obj-fill-258:head# 2020-09-12T14:35:13.558 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.599 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ec0002dd:::obj-fill-205:head# 2020-09-12T14:35:13.599 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.641 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ec740b5c:::obj-fill-46:head# 2020-09-12T14:35:13.641 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.683 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ed35dda4:::obj-fill-170:head# 2020-09-12T14:35:13.683 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.724 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ed51b8f7:::obj-fill-91:head# 2020-09-12T14:35:13.725 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.766 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:edd96161:::obj-fill-48:head# 2020-09-12T14:35:13.766 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.808 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ee3f0d4b:::obj-fill-47:head# 2020-09-12T14:35:13.808 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.849 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ee4784dd:::obj-fill-480:head# 2020-09-12T14:35:13.850 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.891 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ee5a8271:::obj-fill-198:head# 2020-09-12T14:35:13.891 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.933 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ef745b66:::obj-fill-145:head# 2020-09-12T14:35:13.933 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:13.974 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:efa5309d:::obj-fill-215:head# 2020-09-12T14:35:13.975 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.016 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:efbdd3a1:::obj-fill-276:head# 2020-09-12T14:35:14.016 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.058 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:f059d4c0:::obj-fill-482:head# 2020-09-12T14:35:14.058 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.099 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:f0f9fb7a:::obj-fill-379:head# 2020-09-12T14:35:14.100 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.141 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:f16a3656:::obj-fill-462:head# 2020-09-12T14:35:14.142 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.183 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:f1b395ae:::obj-fill-25:head# 2020-09-12T14:35:14.183 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.224 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:f5876dad:::obj-fill-359:head# 2020-09-12T14:35:14.225 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.266 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:f5a283cd:::obj-fill-534:head# 2020-09-12T14:35:14.267 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.308 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:f5cd905a:::obj-fill-358:head# 2020-09-12T14:35:14.308 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.372 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:f5e48302:::obj-fill-406:head# 2020-09-12T14:35:14.372 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.433 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:f5ea65fc:::obj-fill-52:head# 2020-09-12T14:35:14.433 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.466 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:f642de6b:::obj-fill-204:head# 2020-09-12T14:35:14.467 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.500 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:f716eafd:::obj-fill-502:head# 2020-09-12T14:35:14.500 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.533 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:f752e9cb:::obj-fill-23:head# 2020-09-12T14:35:14.533 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.566 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:f85fe42c:::obj-fill-346:head# 2020-09-12T14:35:14.567 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.600 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:f882fd3b:::obj-fill-355:head# 2020-09-12T14:35:14.600 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.682 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:fa133332:::obj-fill-309:head# 2020-09-12T14:35:14.682 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.716 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:fa875e52:::obj-fill-64:head# 2020-09-12T14:35:14.717 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.750 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:fad91a22:::obj-fill-481:head# 2020-09-12T14:35:14.750 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.787 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:fbbc3122:::obj-fill-111:head# 2020-09-12T14:35:14.788 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.816 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:fc539ddb:::obj-fill-181:head# 2020-09-12T14:35:14.817 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.850 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:fc54128d:::obj-fill-104:head# 2020-09-12T14:35:14.850 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.883 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:fe00aded:::obj-fill-365:head# 2020-09-12T14:35:14.883 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.917 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:fe1e9971:::obj-fill-189:head# 2020-09-12T14:35:14.917 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.950 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ff0e0563:::obj-fill-508:head# 2020-09-12T14:35:14.950 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:14.983 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ff4e156c:::obj-fill-441:head# 2020-09-12T14:35:14.983 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:15.016 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:ffa355f7:::obj-fill-500:head# 2020-09-12T14:35:15.017 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:15.050 INFO:tasks.workunit.client.0.smithi027.stdout:Write #1:fff6c93b:::obj-fill-277:head# 2020-09-12T14:35:15.050 INFO:tasks.workunit.client.0.smithi027.stdout:snapset 0=[]:{} 2020-09-12T14:35:15.083 INFO:tasks.workunit.client.0.smithi027.stdout:write_pg epoch 34 info 1.0( v 34'600 (34'500,34'600] local-lis/les=24/25 n=600 ec=21/21 lis/c=24/24 les/c/f=25/25/0 sis=24) 2020-09-12T14:35:15.125 INFO:tasks.workunit.client.0.smithi027.stdout:Import successful 2020-09-12T14:35:15.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:494: TEST_backfill_multi_partial: ceph pg dump pgs 2020-09-12T14:35:16.033 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:35:16.034 INFO:tasks.workunit.client.0.smithi027.stdout:3.0 0 0 0 0 0 0 0 0 0 0 active+clean 2020-09-12T14:34:41.265812+0000 0'0 44:43 [2,0] 2 [2,0] 2 0'0 2020-09-12T14:32:03.946070+0000 0'0 2020-09-12T14:32:03.946070+0000 0 2020-09-12T14:35:16.035 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 542 0 0 0 0 1415162 0 0 42 42 stale+active+clean 2020-09-12T14:32:01.131349+0000 34'542 34:569 [1] 1 [1] 1 0'0 2020-09-12T14:31:58.829524+0000 0'0 2020-09-12T14:31:58.829524+0000 0 2020-09-12T14:35:16.035 INFO:tasks.workunit.client.0.smithi027.stdout:2.0 600 0 0 0 0 614400 0 0 100 100 active+clean 2020-09-12T14:34:41.223901+0000 39'600 44:655 [2,0] 2 [2,0] 2 0'0 2020-09-12T14:32:01.404494+0000 0'0 2020-09-12T14:32:01.404494+0000 0 2020-09-12T14:35:16.036 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:35:16.036 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:35:16.037 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:35:16.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:495: TEST_backfill_multi_partial: sleep 20 2020-09-12T14:35:36.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:496: TEST_backfill_multi_partial: ceph pg dump pgs 2020-09-12T14:35:36.438 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:35:36.438 INFO:tasks.workunit.client.0.smithi027.stdout:3.0 0 0 0 0 0 0 0 0 0 0 active+clean 2020-09-12T14:34:41.265812+0000 0'0 44:43 [2,0] 2 [2,0] 2 0'0 2020-09-12T14:32:03.946070+0000 0'0 2020-09-12T14:32:03.946070+0000 0 2020-09-12T14:35:36.439 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 542 0 0 0 0 1415162 0 0 42 42 stale+active+clean 2020-09-12T14:32:01.131349+0000 34'542 34:569 [1] 1 [1] 1 0'0 2020-09-12T14:31:58.829524+0000 0'0 2020-09-12T14:31:58.829524+0000 0 2020-09-12T14:35:36.439 INFO:tasks.workunit.client.0.smithi027.stdout:2.0 600 0 0 0 0 614400 0 0 100 100 active+clean 2020-09-12T14:34:41.223901+0000 39'600 44:655 [2,0] 2 [2,0] 2 0'0 2020-09-12T14:32:01.404494+0000 0'0 2020-09-12T14:32:01.404494+0000 0 2020-09-12T14:35:36.439 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:35:36.439 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:35:36.440 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:35:36.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:499: TEST_backfill_multi_partial: dd if=/dev/urandom of=td/osd-backfill-space/datafile bs=2611 count=1 2020-09-12T14:35:36.450 INFO:tasks.workunit.client.0.smithi027.stderr:1+0 records in 2020-09-12T14:35:36.450 INFO:tasks.workunit.client.0.smithi027.stderr:1+0 records out 2020-09-12T14:35:36.450 INFO:tasks.workunit.client.0.smithi027.stderr:2611 bytes (2.6 kB, 2.5 KiB) copied, 8.9258e-05 s, 29.3 MB/s 2020-09-12T14:35:36.451 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: seq 1 600 2020-09-12T14:35:36.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:36.452 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:36.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:36.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-1 td/osd-backfill-space/datafile 2020-09-12T14:35:36.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:36.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-1 td/osd-backfill-space/datafile 2020-09-12T14:35:36.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:36.684 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:36.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:36.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-2 td/osd-backfill-space/datafile 2020-09-12T14:35:36.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:36.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-2 td/osd-backfill-space/datafile 2020-09-12T14:35:36.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:36.910 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:36.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:36.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-3 td/osd-backfill-space/datafile 2020-09-12T14:35:37.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:37.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-3 td/osd-backfill-space/datafile 2020-09-12T14:35:37.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:37.136 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:37.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:37.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-4 td/osd-backfill-space/datafile 2020-09-12T14:35:37.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:37.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-4 td/osd-backfill-space/datafile 2020-09-12T14:35:37.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:37.362 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:37.362 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:37.363 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-5 td/osd-backfill-space/datafile 2020-09-12T14:35:37.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:37.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-5 td/osd-backfill-space/datafile 2020-09-12T14:35:37.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:37.578 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:37.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:37.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-6 td/osd-backfill-space/datafile 2020-09-12T14:35:37.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:37.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-6 td/osd-backfill-space/datafile 2020-09-12T14:35:37.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:37.795 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:37.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:37.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-7 td/osd-backfill-space/datafile 2020-09-12T14:35:37.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:37.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-7 td/osd-backfill-space/datafile 2020-09-12T14:35:38.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:38.030 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:38.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:38.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-8 td/osd-backfill-space/datafile 2020-09-12T14:35:38.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:38.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-8 td/osd-backfill-space/datafile 2020-09-12T14:35:38.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:38.248 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:38.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:38.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-9 td/osd-backfill-space/datafile 2020-09-12T14:35:38.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:38.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-9 td/osd-backfill-space/datafile 2020-09-12T14:35:38.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:38.474 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:38.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:38.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-10 td/osd-backfill-space/datafile 2020-09-12T14:35:38.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:38.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-10 td/osd-backfill-space/datafile 2020-09-12T14:35:38.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:38.725 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:38.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:38.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-11 td/osd-backfill-space/datafile 2020-09-12T14:35:38.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:38.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-11 td/osd-backfill-space/datafile 2020-09-12T14:35:38.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:38.943 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:38.943 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:38.944 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-12 td/osd-backfill-space/datafile 2020-09-12T14:35:39.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:39.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-12 td/osd-backfill-space/datafile 2020-09-12T14:35:39.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:39.169 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:39.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:39.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-13 td/osd-backfill-space/datafile 2020-09-12T14:35:39.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:39.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-13 td/osd-backfill-space/datafile 2020-09-12T14:35:39.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:39.394 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:39.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:39.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-14 td/osd-backfill-space/datafile 2020-09-12T14:35:39.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:39.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-14 td/osd-backfill-space/datafile 2020-09-12T14:35:39.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:39.621 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:39.621 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:39.621 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-15 td/osd-backfill-space/datafile 2020-09-12T14:35:39.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:39.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-15 td/osd-backfill-space/datafile 2020-09-12T14:35:39.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:39.839 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:39.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:39.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-16 td/osd-backfill-space/datafile 2020-09-12T14:35:39.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:39.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-16 td/osd-backfill-space/datafile 2020-09-12T14:35:40.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:40.056 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:40.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:40.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-17 td/osd-backfill-space/datafile 2020-09-12T14:35:40.164 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:40.164 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-17 td/osd-backfill-space/datafile 2020-09-12T14:35:40.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:40.282 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:40.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:40.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-18 td/osd-backfill-space/datafile 2020-09-12T14:35:40.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:40.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-18 td/osd-backfill-space/datafile 2020-09-12T14:35:40.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:40.550 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:40.550 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:40.551 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-19 td/osd-backfill-space/datafile 2020-09-12T14:35:40.671 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:40.672 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-19 td/osd-backfill-space/datafile 2020-09-12T14:35:40.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:40.781 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:40.782 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:40.782 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-20 td/osd-backfill-space/datafile 2020-09-12T14:35:40.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:40.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-20 td/osd-backfill-space/datafile 2020-09-12T14:35:40.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:40.999 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:41.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:41.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-21 td/osd-backfill-space/datafile 2020-09-12T14:35:41.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:41.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-21 td/osd-backfill-space/datafile 2020-09-12T14:35:41.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:41.213 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:41.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:41.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-22 td/osd-backfill-space/datafile 2020-09-12T14:35:41.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:41.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-22 td/osd-backfill-space/datafile 2020-09-12T14:35:41.438 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:41.439 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:41.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:41.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-23 td/osd-backfill-space/datafile 2020-09-12T14:35:41.622 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:41.622 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-23 td/osd-backfill-space/datafile 2020-09-12T14:35:41.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:41.728 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:41.728 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:41.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-24 td/osd-backfill-space/datafile 2020-09-12T14:35:41.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:41.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-24 td/osd-backfill-space/datafile 2020-09-12T14:35:41.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:41.937 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:41.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:41.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-25 td/osd-backfill-space/datafile 2020-09-12T14:35:42.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:42.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-25 td/osd-backfill-space/datafile 2020-09-12T14:35:42.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:42.155 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:42.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:42.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-26 td/osd-backfill-space/datafile 2020-09-12T14:35:42.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:42.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-26 td/osd-backfill-space/datafile 2020-09-12T14:35:42.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:42.398 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:42.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:42.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-27 td/osd-backfill-space/datafile 2020-09-12T14:35:42.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:42.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-27 td/osd-backfill-space/datafile 2020-09-12T14:35:42.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:42.627 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:42.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:42.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-28 td/osd-backfill-space/datafile 2020-09-12T14:35:42.766 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:42.766 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-28 td/osd-backfill-space/datafile 2020-09-12T14:35:42.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:42.875 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:42.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:42.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-29 td/osd-backfill-space/datafile 2020-09-12T14:35:42.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:42.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-29 td/osd-backfill-space/datafile 2020-09-12T14:35:43.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:43.093 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:43.093 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:43.093 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-30 td/osd-backfill-space/datafile 2020-09-12T14:35:43.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:43.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-30 td/osd-backfill-space/datafile 2020-09-12T14:35:43.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:43.332 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:43.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:43.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-31 td/osd-backfill-space/datafile 2020-09-12T14:35:43.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:43.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-31 td/osd-backfill-space/datafile 2020-09-12T14:35:43.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:43.567 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:43.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:43.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-32 td/osd-backfill-space/datafile 2020-09-12T14:35:43.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:43.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-32 td/osd-backfill-space/datafile 2020-09-12T14:35:43.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:43.801 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:43.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:43.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-33 td/osd-backfill-space/datafile 2020-09-12T14:35:43.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:43.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-33 td/osd-backfill-space/datafile 2020-09-12T14:35:44.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:44.018 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:44.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:44.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-34 td/osd-backfill-space/datafile 2020-09-12T14:35:44.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:44.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-34 td/osd-backfill-space/datafile 2020-09-12T14:35:44.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:44.252 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:44.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:44.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-35 td/osd-backfill-space/datafile 2020-09-12T14:35:44.378 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:44.378 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-35 td/osd-backfill-space/datafile 2020-09-12T14:35:44.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:44.504 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:44.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:44.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-36 td/osd-backfill-space/datafile 2020-09-12T14:35:44.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:44.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-36 td/osd-backfill-space/datafile 2020-09-12T14:35:44.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:44.755 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:44.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:44.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-37 td/osd-backfill-space/datafile 2020-09-12T14:35:44.871 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:44.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-37 td/osd-backfill-space/datafile 2020-09-12T14:35:44.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:44.989 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:44.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:44.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-38 td/osd-backfill-space/datafile 2020-09-12T14:35:45.106 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:45.106 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-38 td/osd-backfill-space/datafile 2020-09-12T14:35:45.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:45.223 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:45.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:45.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-39 td/osd-backfill-space/datafile 2020-09-12T14:35:45.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:45.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-39 td/osd-backfill-space/datafile 2020-09-12T14:35:45.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:45.491 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:45.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:45.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-40 td/osd-backfill-space/datafile 2020-09-12T14:35:45.616 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:45.617 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-40 td/osd-backfill-space/datafile 2020-09-12T14:35:45.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:45.734 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:45.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:45.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-41 td/osd-backfill-space/datafile 2020-09-12T14:35:45.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:45.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-41 td/osd-backfill-space/datafile 2020-09-12T14:35:45.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:45.960 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:45.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:45.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-42 td/osd-backfill-space/datafile 2020-09-12T14:35:46.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:46.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-42 td/osd-backfill-space/datafile 2020-09-12T14:35:46.177 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:46.178 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:46.178 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:46.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-43 td/osd-backfill-space/datafile 2020-09-12T14:35:46.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:46.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-43 td/osd-backfill-space/datafile 2020-09-12T14:35:46.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:46.437 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:46.437 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:46.437 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-44 td/osd-backfill-space/datafile 2020-09-12T14:35:46.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:46.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-44 td/osd-backfill-space/datafile 2020-09-12T14:35:46.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:46.730 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:46.730 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:46.730 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-45 td/osd-backfill-space/datafile 2020-09-12T14:35:46.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:46.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-45 td/osd-backfill-space/datafile 2020-09-12T14:35:46.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:46.964 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:46.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:46.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-46 td/osd-backfill-space/datafile 2020-09-12T14:35:47.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:47.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-46 td/osd-backfill-space/datafile 2020-09-12T14:35:47.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:47.199 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:47.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:47.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-47 td/osd-backfill-space/datafile 2020-09-12T14:35:47.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:47.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-47 td/osd-backfill-space/datafile 2020-09-12T14:35:47.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:47.443 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:47.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:47.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-48 td/osd-backfill-space/datafile 2020-09-12T14:35:47.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:47.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-48 td/osd-backfill-space/datafile 2020-09-12T14:35:47.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:47.693 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:47.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:47.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-49 td/osd-backfill-space/datafile 2020-09-12T14:35:47.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:47.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-49 td/osd-backfill-space/datafile 2020-09-12T14:35:47.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:47.911 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:47.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:47.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-50 td/osd-backfill-space/datafile 2020-09-12T14:35:48.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:48.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-50 td/osd-backfill-space/datafile 2020-09-12T14:35:48.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:48.128 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:48.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:48.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-51 td/osd-backfill-space/datafile 2020-09-12T14:35:48.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:48.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-51 td/osd-backfill-space/datafile 2020-09-12T14:35:48.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:48.371 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:48.371 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:48.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-52 td/osd-backfill-space/datafile 2020-09-12T14:35:48.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:48.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-52 td/osd-backfill-space/datafile 2020-09-12T14:35:48.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:48.605 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:48.606 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:48.606 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-53 td/osd-backfill-space/datafile 2020-09-12T14:35:48.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:48.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-53 td/osd-backfill-space/datafile 2020-09-12T14:35:48.864 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:48.865 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:48.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:48.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-54 td/osd-backfill-space/datafile 2020-09-12T14:35:48.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:48.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-54 td/osd-backfill-space/datafile 2020-09-12T14:35:49.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:49.099 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:49.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:49.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-55 td/osd-backfill-space/datafile 2020-09-12T14:35:49.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:49.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-55 td/osd-backfill-space/datafile 2020-09-12T14:35:49.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:49.359 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:49.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:49.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-56 td/osd-backfill-space/datafile 2020-09-12T14:35:49.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:49.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-56 td/osd-backfill-space/datafile 2020-09-12T14:35:49.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:49.594 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:49.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:49.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-57 td/osd-backfill-space/datafile 2020-09-12T14:35:49.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:49.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-57 td/osd-backfill-space/datafile 2020-09-12T14:35:49.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:49.819 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:49.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:49.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-58 td/osd-backfill-space/datafile 2020-09-12T14:35:49.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:49.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-58 td/osd-backfill-space/datafile 2020-09-12T14:35:50.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:50.053 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:50.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:50.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-59 td/osd-backfill-space/datafile 2020-09-12T14:35:50.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:50.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-59 td/osd-backfill-space/datafile 2020-09-12T14:35:50.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:50.271 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:50.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:50.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-60 td/osd-backfill-space/datafile 2020-09-12T14:35:50.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:50.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-60 td/osd-backfill-space/datafile 2020-09-12T14:35:50.547 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:50.547 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:50.548 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:50.548 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-61 td/osd-backfill-space/datafile 2020-09-12T14:35:50.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:50.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-61 td/osd-backfill-space/datafile 2020-09-12T14:35:50.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:50.807 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:50.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:50.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-62 td/osd-backfill-space/datafile 2020-09-12T14:35:50.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:50.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-62 td/osd-backfill-space/datafile 2020-09-12T14:35:51.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:51.057 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:51.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:51.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-63 td/osd-backfill-space/datafile 2020-09-12T14:35:51.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:51.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-63 td/osd-backfill-space/datafile 2020-09-12T14:35:51.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:51.292 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:51.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:51.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-64 td/osd-backfill-space/datafile 2020-09-12T14:35:51.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:51.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-64 td/osd-backfill-space/datafile 2020-09-12T14:35:51.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:51.535 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:51.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:51.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-65 td/osd-backfill-space/datafile 2020-09-12T14:35:51.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:51.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-65 td/osd-backfill-space/datafile 2020-09-12T14:35:51.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:51.769 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:51.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:51.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-66 td/osd-backfill-space/datafile 2020-09-12T14:35:51.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:51.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-66 td/osd-backfill-space/datafile 2020-09-12T14:35:51.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:51.995 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:51.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:51.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-67 td/osd-backfill-space/datafile 2020-09-12T14:35:52.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:52.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-67 td/osd-backfill-space/datafile 2020-09-12T14:35:52.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:52.212 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:52.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:52.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-68 td/osd-backfill-space/datafile 2020-09-12T14:35:52.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:52.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-68 td/osd-backfill-space/datafile 2020-09-12T14:35:52.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:52.455 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:52.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:52.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-69 td/osd-backfill-space/datafile 2020-09-12T14:35:52.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:52.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-69 td/osd-backfill-space/datafile 2020-09-12T14:35:52.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:52.723 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:52.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:52.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-70 td/osd-backfill-space/datafile 2020-09-12T14:35:52.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:52.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-70 td/osd-backfill-space/datafile 2020-09-12T14:35:52.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:52.966 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:52.966 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:52.966 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-71 td/osd-backfill-space/datafile 2020-09-12T14:35:53.091 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:53.091 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-71 td/osd-backfill-space/datafile 2020-09-12T14:35:53.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:53.208 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:53.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:53.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-72 td/osd-backfill-space/datafile 2020-09-12T14:35:53.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:53.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-72 td/osd-backfill-space/datafile 2020-09-12T14:35:53.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:53.443 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:53.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:53.444 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-73 td/osd-backfill-space/datafile 2020-09-12T14:35:53.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:53.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-73 td/osd-backfill-space/datafile 2020-09-12T14:35:53.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:53.702 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:53.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:53.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-74 td/osd-backfill-space/datafile 2020-09-12T14:35:53.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:53.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-74 td/osd-backfill-space/datafile 2020-09-12T14:35:53.928 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:53.928 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:53.929 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:53.929 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-75 td/osd-backfill-space/datafile 2020-09-12T14:35:54.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:54.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-75 td/osd-backfill-space/datafile 2020-09-12T14:35:54.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:54.146 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:54.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:54.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-76 td/osd-backfill-space/datafile 2020-09-12T14:35:54.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:54.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-76 td/osd-backfill-space/datafile 2020-09-12T14:35:54.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:54.380 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:54.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:54.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-77 td/osd-backfill-space/datafile 2020-09-12T14:35:54.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:54.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-77 td/osd-backfill-space/datafile 2020-09-12T14:35:54.597 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:54.598 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:54.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:54.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-78 td/osd-backfill-space/datafile 2020-09-12T14:35:54.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:54.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-78 td/osd-backfill-space/datafile 2020-09-12T14:35:54.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:54.832 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:54.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:54.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-79 td/osd-backfill-space/datafile 2020-09-12T14:35:54.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:54.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-79 td/osd-backfill-space/datafile 2020-09-12T14:35:55.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:55.059 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:55.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:55.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-80 td/osd-backfill-space/datafile 2020-09-12T14:35:55.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:55.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-80 td/osd-backfill-space/datafile 2020-09-12T14:35:55.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:55.276 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:55.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:55.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-81 td/osd-backfill-space/datafile 2020-09-12T14:35:55.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:55.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-81 td/osd-backfill-space/datafile 2020-09-12T14:35:55.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:55.494 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:55.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:55.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-82 td/osd-backfill-space/datafile 2020-09-12T14:35:55.644 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:55.644 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-82 td/osd-backfill-space/datafile 2020-09-12T14:35:55.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:55.753 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:55.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:55.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-83 td/osd-backfill-space/datafile 2020-09-12T14:35:55.862 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:55.862 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-83 td/osd-backfill-space/datafile 2020-09-12T14:35:55.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:55.971 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:55.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:55.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-84 td/osd-backfill-space/datafile 2020-09-12T14:35:56.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:56.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-84 td/osd-backfill-space/datafile 2020-09-12T14:35:56.196 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:56.197 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:56.197 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:56.197 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-85 td/osd-backfill-space/datafile 2020-09-12T14:35:56.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:56.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-85 td/osd-backfill-space/datafile 2020-09-12T14:35:56.439 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:56.439 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:56.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:56.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-86 td/osd-backfill-space/datafile 2020-09-12T14:35:56.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:56.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-86 td/osd-backfill-space/datafile 2020-09-12T14:35:56.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:56.716 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:56.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:56.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-87 td/osd-backfill-space/datafile 2020-09-12T14:35:56.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:56.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-87 td/osd-backfill-space/datafile 2020-09-12T14:35:56.933 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:56.934 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:56.934 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:56.934 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-88 td/osd-backfill-space/datafile 2020-09-12T14:35:57.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:57.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-88 td/osd-backfill-space/datafile 2020-09-12T14:35:57.151 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:57.152 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:57.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:57.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-89 td/osd-backfill-space/datafile 2020-09-12T14:35:57.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:57.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-89 td/osd-backfill-space/datafile 2020-09-12T14:35:57.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:57.369 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:57.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:57.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-90 td/osd-backfill-space/datafile 2020-09-12T14:35:57.477 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:57.477 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-90 td/osd-backfill-space/datafile 2020-09-12T14:35:57.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:57.578 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:57.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:57.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-91 td/osd-backfill-space/datafile 2020-09-12T14:35:57.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:57.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-91 td/osd-backfill-space/datafile 2020-09-12T14:35:57.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:57.796 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:57.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:57.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-92 td/osd-backfill-space/datafile 2020-09-12T14:35:57.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:57.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-92 td/osd-backfill-space/datafile 2020-09-12T14:35:58.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:58.013 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:58.014 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:58.014 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-93 td/osd-backfill-space/datafile 2020-09-12T14:35:58.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:58.122 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-93 td/osd-backfill-space/datafile 2020-09-12T14:35:58.231 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:58.231 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:58.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:58.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-94 td/osd-backfill-space/datafile 2020-09-12T14:35:58.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:58.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-94 td/osd-backfill-space/datafile 2020-09-12T14:35:58.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:58.474 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:58.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:58.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-95 td/osd-backfill-space/datafile 2020-09-12T14:35:58.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:58.583 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-95 td/osd-backfill-space/datafile 2020-09-12T14:35:58.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:58.726 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:58.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:58.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-96 td/osd-backfill-space/datafile 2020-09-12T14:35:58.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:58.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-96 td/osd-backfill-space/datafile 2020-09-12T14:35:58.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:58.943 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:58.943 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:58.944 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-97 td/osd-backfill-space/datafile 2020-09-12T14:35:59.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:59.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-97 td/osd-backfill-space/datafile 2020-09-12T14:35:59.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:59.161 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:59.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:59.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-98 td/osd-backfill-space/datafile 2020-09-12T14:35:59.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:59.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-98 td/osd-backfill-space/datafile 2020-09-12T14:35:59.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:59.386 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:59.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:59.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-99 td/osd-backfill-space/datafile 2020-09-12T14:35:59.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:59.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-99 td/osd-backfill-space/datafile 2020-09-12T14:35:59.612 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:59.612 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:59.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:59.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-100 td/osd-backfill-space/datafile 2020-09-12T14:35:59.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:59.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-100 td/osd-backfill-space/datafile 2020-09-12T14:35:59.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:35:59.830 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:35:59.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:59.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-101 td/osd-backfill-space/datafile 2020-09-12T14:35:59.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:35:59.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-101 td/osd-backfill-space/datafile 2020-09-12T14:36:00.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:00.047 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:00.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:00.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-102 td/osd-backfill-space/datafile 2020-09-12T14:36:00.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:00.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-102 td/osd-backfill-space/datafile 2020-09-12T14:36:00.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:00.265 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:00.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:00.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-103 td/osd-backfill-space/datafile 2020-09-12T14:36:00.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:00.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-103 td/osd-backfill-space/datafile 2020-09-12T14:36:00.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:00.500 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:00.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:00.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-104 td/osd-backfill-space/datafile 2020-09-12T14:36:00.616 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:00.617 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-104 td/osd-backfill-space/datafile 2020-09-12T14:36:00.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:00.776 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:00.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:00.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-105 td/osd-backfill-space/datafile 2020-09-12T14:36:00.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:00.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-105 td/osd-backfill-space/datafile 2020-09-12T14:36:00.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:00.994 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:00.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:00.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-106 td/osd-backfill-space/datafile 2020-09-12T14:36:01.094 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:01.094 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-106 td/osd-backfill-space/datafile 2020-09-12T14:36:01.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:01.195 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:01.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:01.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-107 td/osd-backfill-space/datafile 2020-09-12T14:36:01.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:01.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-107 td/osd-backfill-space/datafile 2020-09-12T14:36:01.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:01.429 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:01.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:01.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-108 td/osd-backfill-space/datafile 2020-09-12T14:36:01.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:01.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-108 td/osd-backfill-space/datafile 2020-09-12T14:36:01.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:01.647 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:01.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:01.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-109 td/osd-backfill-space/datafile 2020-09-12T14:36:01.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:01.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-109 td/osd-backfill-space/datafile 2020-09-12T14:36:01.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:01.872 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:01.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:01.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-110 td/osd-backfill-space/datafile 2020-09-12T14:36:01.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:01.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-110 td/osd-backfill-space/datafile 2020-09-12T14:36:02.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:02.132 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:02.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:02.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-111 td/osd-backfill-space/datafile 2020-09-12T14:36:02.240 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:02.240 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-111 td/osd-backfill-space/datafile 2020-09-12T14:36:02.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:02.341 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:02.342 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:02.342 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-112 td/osd-backfill-space/datafile 2020-09-12T14:36:02.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:02.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-112 td/osd-backfill-space/datafile 2020-09-12T14:36:02.584 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:02.584 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:02.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:02.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-113 td/osd-backfill-space/datafile 2020-09-12T14:36:02.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:02.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-113 td/osd-backfill-space/datafile 2020-09-12T14:36:02.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:02.844 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:02.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:02.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-114 td/osd-backfill-space/datafile 2020-09-12T14:36:02.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:02.953 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-114 td/osd-backfill-space/datafile 2020-09-12T14:36:03.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:03.061 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:03.062 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:03.062 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-115 td/osd-backfill-space/datafile 2020-09-12T14:36:03.178 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:03.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-115 td/osd-backfill-space/datafile 2020-09-12T14:36:03.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:03.304 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:03.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:03.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-116 td/osd-backfill-space/datafile 2020-09-12T14:36:03.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:03.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-116 td/osd-backfill-space/datafile 2020-09-12T14:36:03.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:03.522 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:03.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:03.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-117 td/osd-backfill-space/datafile 2020-09-12T14:36:03.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:03.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-117 td/osd-backfill-space/datafile 2020-09-12T14:36:03.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:03.748 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:03.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:03.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-118 td/osd-backfill-space/datafile 2020-09-12T14:36:03.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:03.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-118 td/osd-backfill-space/datafile 2020-09-12T14:36:03.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:03.990 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:03.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:03.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-119 td/osd-backfill-space/datafile 2020-09-12T14:36:04.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:04.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-119 td/osd-backfill-space/datafile 2020-09-12T14:36:04.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:04.216 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:04.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:04.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-120 td/osd-backfill-space/datafile 2020-09-12T14:36:04.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:04.342 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-120 td/osd-backfill-space/datafile 2020-09-12T14:36:04.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:04.509 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:04.510 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:04.510 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-121 td/osd-backfill-space/datafile 2020-09-12T14:36:04.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:04.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-121 td/osd-backfill-space/datafile 2020-09-12T14:36:04.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:04.744 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:04.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:04.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-122 td/osd-backfill-space/datafile 2020-09-12T14:36:04.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:04.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-122 td/osd-backfill-space/datafile 2020-09-12T14:36:04.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:04.953 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:04.953 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:04.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-123 td/osd-backfill-space/datafile 2020-09-12T14:36:05.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:05.062 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-123 td/osd-backfill-space/datafile 2020-09-12T14:36:05.178 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:05.179 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:05.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:05.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-124 td/osd-backfill-space/datafile 2020-09-12T14:36:05.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:05.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-124 td/osd-backfill-space/datafile 2020-09-12T14:36:05.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:05.397 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:05.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:05.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-125 td/osd-backfill-space/datafile 2020-09-12T14:36:05.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:05.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-125 td/osd-backfill-space/datafile 2020-09-12T14:36:05.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:05.631 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:05.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:05.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-126 td/osd-backfill-space/datafile 2020-09-12T14:36:05.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:05.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-126 td/osd-backfill-space/datafile 2020-09-12T14:36:05.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:05.866 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:05.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:05.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-127 td/osd-backfill-space/datafile 2020-09-12T14:36:05.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:05.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-127 td/osd-backfill-space/datafile 2020-09-12T14:36:06.091 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:06.092 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:06.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:06.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-128 td/osd-backfill-space/datafile 2020-09-12T14:36:06.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:06.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-128 td/osd-backfill-space/datafile 2020-09-12T14:36:06.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:06.309 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:06.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:06.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-129 td/osd-backfill-space/datafile 2020-09-12T14:36:06.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:06.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-129 td/osd-backfill-space/datafile 2020-09-12T14:36:06.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:06.569 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:06.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:06.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-130 td/osd-backfill-space/datafile 2020-09-12T14:36:06.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:06.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-130 td/osd-backfill-space/datafile 2020-09-12T14:36:06.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:06.795 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:06.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:06.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-131 td/osd-backfill-space/datafile 2020-09-12T14:36:06.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:06.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-131 td/osd-backfill-space/datafile 2020-09-12T14:36:07.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:07.021 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:07.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:07.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-132 td/osd-backfill-space/datafile 2020-09-12T14:36:07.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:07.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-132 td/osd-backfill-space/datafile 2020-09-12T14:36:07.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:07.293 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:07.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:07.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-133 td/osd-backfill-space/datafile 2020-09-12T14:36:07.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:07.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-133 td/osd-backfill-space/datafile 2020-09-12T14:36:07.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:07.498 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:07.498 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:07.498 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-134 td/osd-backfill-space/datafile 2020-09-12T14:36:07.606 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:07.606 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-134 td/osd-backfill-space/datafile 2020-09-12T14:36:07.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:07.715 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:07.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:07.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-135 td/osd-backfill-space/datafile 2020-09-12T14:36:07.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:07.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-135 td/osd-backfill-space/datafile 2020-09-12T14:36:07.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:07.951 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:07.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:07.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-136 td/osd-backfill-space/datafile 2020-09-12T14:36:08.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:08.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-136 td/osd-backfill-space/datafile 2020-09-12T14:36:08.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:08.167 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:08.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:08.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-137 td/osd-backfill-space/datafile 2020-09-12T14:36:08.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:08.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-137 td/osd-backfill-space/datafile 2020-09-12T14:36:08.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:08.393 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:08.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:08.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-138 td/osd-backfill-space/datafile 2020-09-12T14:36:08.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:08.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-138 td/osd-backfill-space/datafile 2020-09-12T14:36:08.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:08.611 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:08.612 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:08.612 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-139 td/osd-backfill-space/datafile 2020-09-12T14:36:08.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:08.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-139 td/osd-backfill-space/datafile 2020-09-12T14:36:08.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:08.854 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:08.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:08.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-140 td/osd-backfill-space/datafile 2020-09-12T14:36:08.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:08.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-140 td/osd-backfill-space/datafile 2020-09-12T14:36:09.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:09.088 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:09.089 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:09.089 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-141 td/osd-backfill-space/datafile 2020-09-12T14:36:09.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:09.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-141 td/osd-backfill-space/datafile 2020-09-12T14:36:09.298 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:09.298 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:09.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:09.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-142 td/osd-backfill-space/datafile 2020-09-12T14:36:09.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:09.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-142 td/osd-backfill-space/datafile 2020-09-12T14:36:09.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:09.523 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:09.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:09.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-143 td/osd-backfill-space/datafile 2020-09-12T14:36:09.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:09.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-143 td/osd-backfill-space/datafile 2020-09-12T14:36:09.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:09.741 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:09.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:09.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-144 td/osd-backfill-space/datafile 2020-09-12T14:36:09.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:09.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-144 td/osd-backfill-space/datafile 2020-09-12T14:36:09.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:09.976 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:09.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:09.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-145 td/osd-backfill-space/datafile 2020-09-12T14:36:10.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:10.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-145 td/osd-backfill-space/datafile 2020-09-12T14:36:10.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:10.177 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:10.177 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:10.177 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-146 td/osd-backfill-space/datafile 2020-09-12T14:36:10.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:10.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-146 td/osd-backfill-space/datafile 2020-09-12T14:36:10.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:10.411 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:10.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:10.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-147 td/osd-backfill-space/datafile 2020-09-12T14:36:10.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:10.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-147 td/osd-backfill-space/datafile 2020-09-12T14:36:10.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:10.637 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:10.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:10.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-148 td/osd-backfill-space/datafile 2020-09-12T14:36:10.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:10.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-148 td/osd-backfill-space/datafile 2020-09-12T14:36:10.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:10.913 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:10.914 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:10.914 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-149 td/osd-backfill-space/datafile 2020-09-12T14:36:11.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:11.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-149 td/osd-backfill-space/datafile 2020-09-12T14:36:11.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:11.131 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:11.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:11.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-150 td/osd-backfill-space/datafile 2020-09-12T14:36:11.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:11.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-150 td/osd-backfill-space/datafile 2020-09-12T14:36:11.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:11.349 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:11.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:11.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-151 td/osd-backfill-space/datafile 2020-09-12T14:36:11.457 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:11.457 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-151 td/osd-backfill-space/datafile 2020-09-12T14:36:11.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:11.566 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:11.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:11.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-152 td/osd-backfill-space/datafile 2020-09-12T14:36:11.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:11.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-152 td/osd-backfill-space/datafile 2020-09-12T14:36:11.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:11.801 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:11.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:11.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-153 td/osd-backfill-space/datafile 2020-09-12T14:36:11.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:11.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-153 td/osd-backfill-space/datafile 2020-09-12T14:36:12.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:12.019 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:12.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:12.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-154 td/osd-backfill-space/datafile 2020-09-12T14:36:12.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:12.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-154 td/osd-backfill-space/datafile 2020-09-12T14:36:12.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:12.253 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:12.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:12.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-155 td/osd-backfill-space/datafile 2020-09-12T14:36:12.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:12.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-155 td/osd-backfill-space/datafile 2020-09-12T14:36:12.762 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:12.762 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:12.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:12.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-156 td/osd-backfill-space/datafile 2020-09-12T14:36:12.871 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:12.871 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-156 td/osd-backfill-space/datafile 2020-09-12T14:36:12.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:12.980 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:12.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:12.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-157 td/osd-backfill-space/datafile 2020-09-12T14:36:13.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:13.089 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-157 td/osd-backfill-space/datafile 2020-09-12T14:36:13.197 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:13.198 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:13.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:13.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-158 td/osd-backfill-space/datafile 2020-09-12T14:36:13.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:13.315 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-158 td/osd-backfill-space/datafile 2020-09-12T14:36:13.423 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:13.424 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:13.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:13.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-159 td/osd-backfill-space/datafile 2020-09-12T14:36:13.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:13.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-159 td/osd-backfill-space/datafile 2020-09-12T14:36:13.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:13.658 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:13.659 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:13.659 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-160 td/osd-backfill-space/datafile 2020-09-12T14:36:13.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:13.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-160 td/osd-backfill-space/datafile 2020-09-12T14:36:13.894 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:13.894 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:13.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:13.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-161 td/osd-backfill-space/datafile 2020-09-12T14:36:14.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:14.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-161 td/osd-backfill-space/datafile 2020-09-12T14:36:14.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:14.128 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:14.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:14.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-162 td/osd-backfill-space/datafile 2020-09-12T14:36:14.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:14.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-162 td/osd-backfill-space/datafile 2020-09-12T14:36:14.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:14.338 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:14.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:14.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-163 td/osd-backfill-space/datafile 2020-09-12T14:36:14.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:14.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-163 td/osd-backfill-space/datafile 2020-09-12T14:36:14.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:14.572 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:14.573 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:14.573 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-164 td/osd-backfill-space/datafile 2020-09-12T14:36:14.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:14.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-164 td/osd-backfill-space/datafile 2020-09-12T14:36:14.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:14.807 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:14.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:14.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-165 td/osd-backfill-space/datafile 2020-09-12T14:36:14.915 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:14.915 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-165 td/osd-backfill-space/datafile 2020-09-12T14:36:15.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:15.016 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:15.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:15.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-166 td/osd-backfill-space/datafile 2020-09-12T14:36:15.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:15.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-166 td/osd-backfill-space/datafile 2020-09-12T14:36:15.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:15.242 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:15.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:15.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-167 td/osd-backfill-space/datafile 2020-09-12T14:36:15.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:15.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-167 td/osd-backfill-space/datafile 2020-09-12T14:36:15.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:15.477 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:15.477 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:15.477 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-168 td/osd-backfill-space/datafile 2020-09-12T14:36:15.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:15.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-168 td/osd-backfill-space/datafile 2020-09-12T14:36:15.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:15.727 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:15.728 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:15.728 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-169 td/osd-backfill-space/datafile 2020-09-12T14:36:15.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:15.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-169 td/osd-backfill-space/datafile 2020-09-12T14:36:15.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:15.962 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:15.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:15.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-170 td/osd-backfill-space/datafile 2020-09-12T14:36:16.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:16.079 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-170 td/osd-backfill-space/datafile 2020-09-12T14:36:16.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:16.188 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:16.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:16.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-171 td/osd-backfill-space/datafile 2020-09-12T14:36:16.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:16.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-171 td/osd-backfill-space/datafile 2020-09-12T14:36:16.438 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:16.439 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:16.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:16.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-172 td/osd-backfill-space/datafile 2020-09-12T14:36:16.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:16.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-172 td/osd-backfill-space/datafile 2020-09-12T14:36:16.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:16.681 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:16.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:16.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-173 td/osd-backfill-space/datafile 2020-09-12T14:36:16.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:16.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-173 td/osd-backfill-space/datafile 2020-09-12T14:36:16.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:16.908 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:16.908 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:16.908 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-174 td/osd-backfill-space/datafile 2020-09-12T14:36:17.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:17.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-174 td/osd-backfill-space/datafile 2020-09-12T14:36:17.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:17.133 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:17.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:17.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-175 td/osd-backfill-space/datafile 2020-09-12T14:36:17.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:17.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-175 td/osd-backfill-space/datafile 2020-09-12T14:36:17.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:17.376 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:17.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:17.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-176 td/osd-backfill-space/datafile 2020-09-12T14:36:17.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:17.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-176 td/osd-backfill-space/datafile 2020-09-12T14:36:17.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:17.611 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:17.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:17.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-177 td/osd-backfill-space/datafile 2020-09-12T14:36:17.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:17.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-177 td/osd-backfill-space/datafile 2020-09-12T14:36:17.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:17.853 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:17.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:17.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-178 td/osd-backfill-space/datafile 2020-09-12T14:36:18.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:18.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-178 td/osd-backfill-space/datafile 2020-09-12T14:36:18.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:18.146 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:18.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:18.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-179 td/osd-backfill-space/datafile 2020-09-12T14:36:18.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:18.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-179 td/osd-backfill-space/datafile 2020-09-12T14:36:18.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:18.380 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:18.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:18.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-180 td/osd-backfill-space/datafile 2020-09-12T14:36:18.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:18.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-180 td/osd-backfill-space/datafile 2020-09-12T14:36:18.606 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:18.606 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:18.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:18.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-181 td/osd-backfill-space/datafile 2020-09-12T14:36:18.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:18.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-181 td/osd-backfill-space/datafile 2020-09-12T14:36:18.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:18.849 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:18.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:18.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-182 td/osd-backfill-space/datafile 2020-09-12T14:36:18.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:18.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-182 td/osd-backfill-space/datafile 2020-09-12T14:36:19.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:19.075 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:19.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:19.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-183 td/osd-backfill-space/datafile 2020-09-12T14:36:19.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:19.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-183 td/osd-backfill-space/datafile 2020-09-12T14:36:19.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:19.318 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:19.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:19.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-184 td/osd-backfill-space/datafile 2020-09-12T14:36:19.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:19.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-184 td/osd-backfill-space/datafile 2020-09-12T14:36:19.552 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:19.552 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:19.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:19.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-185 td/osd-backfill-space/datafile 2020-09-12T14:36:19.669 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:19.669 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-185 td/osd-backfill-space/datafile 2020-09-12T14:36:19.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:19.787 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:19.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:19.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-186 td/osd-backfill-space/datafile 2020-09-12T14:36:19.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:19.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-186 td/osd-backfill-space/datafile 2020-09-12T14:36:20.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:20.013 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:20.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:20.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-187 td/osd-backfill-space/datafile 2020-09-12T14:36:20.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:20.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-187 td/osd-backfill-space/datafile 2020-09-12T14:36:20.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:20.230 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:20.231 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:20.231 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-188 td/osd-backfill-space/datafile 2020-09-12T14:36:20.347 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:20.347 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-188 td/osd-backfill-space/datafile 2020-09-12T14:36:20.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:20.473 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:20.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:20.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-189 td/osd-backfill-space/datafile 2020-09-12T14:36:20.590 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:20.590 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-189 td/osd-backfill-space/datafile 2020-09-12T14:36:20.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:20.758 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:20.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:20.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-190 td/osd-backfill-space/datafile 2020-09-12T14:36:20.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:20.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-190 td/osd-backfill-space/datafile 2020-09-12T14:36:20.966 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:20.967 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:20.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:20.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-191 td/osd-backfill-space/datafile 2020-09-12T14:36:21.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:21.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-191 td/osd-backfill-space/datafile 2020-09-12T14:36:21.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:21.185 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:21.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:21.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-192 td/osd-backfill-space/datafile 2020-09-12T14:36:21.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:21.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-192 td/osd-backfill-space/datafile 2020-09-12T14:36:21.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:21.420 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:21.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:21.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-193 td/osd-backfill-space/datafile 2020-09-12T14:36:21.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:21.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-193 td/osd-backfill-space/datafile 2020-09-12T14:36:21.653 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:21.653 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:21.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:21.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-194 td/osd-backfill-space/datafile 2020-09-12T14:36:21.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:21.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-194 td/osd-backfill-space/datafile 2020-09-12T14:36:21.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:21.879 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:21.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:21.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-195 td/osd-backfill-space/datafile 2020-09-12T14:36:21.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:21.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-195 td/osd-backfill-space/datafile 2020-09-12T14:36:22.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:22.097 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:22.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:22.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-196 td/osd-backfill-space/datafile 2020-09-12T14:36:22.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:22.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-196 td/osd-backfill-space/datafile 2020-09-12T14:36:22.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:22.323 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:22.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:22.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-197 td/osd-backfill-space/datafile 2020-09-12T14:36:22.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:22.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-197 td/osd-backfill-space/datafile 2020-09-12T14:36:22.590 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:22.591 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:22.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:22.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-198 td/osd-backfill-space/datafile 2020-09-12T14:36:22.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:22.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-198 td/osd-backfill-space/datafile 2020-09-12T14:36:22.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:22.834 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:22.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:22.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-199 td/osd-backfill-space/datafile 2020-09-12T14:36:22.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:22.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-199 td/osd-backfill-space/datafile 2020-09-12T14:36:23.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:23.060 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:23.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:23.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-200 td/osd-backfill-space/datafile 2020-09-12T14:36:23.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:23.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-200 td/osd-backfill-space/datafile 2020-09-12T14:36:23.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:23.303 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:23.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:23.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-201 td/osd-backfill-space/datafile 2020-09-12T14:36:23.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:23.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-201 td/osd-backfill-space/datafile 2020-09-12T14:36:23.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:23.537 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:23.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:23.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-202 td/osd-backfill-space/datafile 2020-09-12T14:36:23.662 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:23.662 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-202 td/osd-backfill-space/datafile 2020-09-12T14:36:23.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:23.788 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:23.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:23.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-203 td/osd-backfill-space/datafile 2020-09-12T14:36:23.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:23.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-203 td/osd-backfill-space/datafile 2020-09-12T14:36:24.038 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:24.039 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:24.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:24.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-204 td/osd-backfill-space/datafile 2020-09-12T14:36:24.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:24.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-204 td/osd-backfill-space/datafile 2020-09-12T14:36:24.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:24.273 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:24.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:24.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-205 td/osd-backfill-space/datafile 2020-09-12T14:36:24.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:24.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-205 td/osd-backfill-space/datafile 2020-09-12T14:36:24.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:24.524 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:24.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:24.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-206 td/osd-backfill-space/datafile 2020-09-12T14:36:24.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:24.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-206 td/osd-backfill-space/datafile 2020-09-12T14:36:24.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:24.742 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:24.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:24.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-207 td/osd-backfill-space/datafile 2020-09-12T14:36:24.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:24.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-207 td/osd-backfill-space/datafile 2020-09-12T14:36:24.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:24.952 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:24.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:24.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-208 td/osd-backfill-space/datafile 2020-09-12T14:36:25.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:25.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-208 td/osd-backfill-space/datafile 2020-09-12T14:36:25.177 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:25.177 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:25.178 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:25.178 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-209 td/osd-backfill-space/datafile 2020-09-12T14:36:25.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:25.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-209 td/osd-backfill-space/datafile 2020-09-12T14:36:25.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:25.412 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:25.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:25.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-210 td/osd-backfill-space/datafile 2020-09-12T14:36:25.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:25.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-210 td/osd-backfill-space/datafile 2020-09-12T14:36:25.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:25.629 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:25.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:25.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-211 td/osd-backfill-space/datafile 2020-09-12T14:36:25.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:25.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-211 td/osd-backfill-space/datafile 2020-09-12T14:36:25.871 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:25.872 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:25.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:25.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-212 td/osd-backfill-space/datafile 2020-09-12T14:36:25.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:25.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-212 td/osd-backfill-space/datafile 2020-09-12T14:36:26.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:26.073 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:26.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:26.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-213 td/osd-backfill-space/datafile 2020-09-12T14:36:26.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:26.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-213 td/osd-backfill-space/datafile 2020-09-12T14:36:26.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:26.282 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:26.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:26.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-214 td/osd-backfill-space/datafile 2020-09-12T14:36:26.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:26.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-214 td/osd-backfill-space/datafile 2020-09-12T14:36:26.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:26.483 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:26.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:26.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-215 td/osd-backfill-space/datafile 2020-09-12T14:36:26.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:26.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-215 td/osd-backfill-space/datafile 2020-09-12T14:36:26.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:26.760 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:26.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:26.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-216 td/osd-backfill-space/datafile 2020-09-12T14:36:26.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:26.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-216 td/osd-backfill-space/datafile 2020-09-12T14:36:26.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:26.977 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:26.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:26.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-217 td/osd-backfill-space/datafile 2020-09-12T14:36:27.094 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:27.094 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-217 td/osd-backfill-space/datafile 2020-09-12T14:36:27.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:27.203 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:27.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:27.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-218 td/osd-backfill-space/datafile 2020-09-12T14:36:27.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:27.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-218 td/osd-backfill-space/datafile 2020-09-12T14:36:27.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:27.429 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:27.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:27.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-219 td/osd-backfill-space/datafile 2020-09-12T14:36:27.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:27.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-219 td/osd-backfill-space/datafile 2020-09-12T14:36:27.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:27.630 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:27.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:27.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-220 td/osd-backfill-space/datafile 2020-09-12T14:36:27.730 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:27.730 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-220 td/osd-backfill-space/datafile 2020-09-12T14:36:27.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:27.831 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:27.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:27.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-221 td/osd-backfill-space/datafile 2020-09-12T14:36:27.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:27.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-221 td/osd-backfill-space/datafile 2020-09-12T14:36:28.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:28.057 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:28.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:28.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-222 td/osd-backfill-space/datafile 2020-09-12T14:36:28.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:28.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-222 td/osd-backfill-space/datafile 2020-09-12T14:36:28.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:28.283 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:28.284 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:28.284 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-223 td/osd-backfill-space/datafile 2020-09-12T14:36:28.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:28.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-223 td/osd-backfill-space/datafile 2020-09-12T14:36:28.508 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:28.509 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:28.510 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:28.510 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-224 td/osd-backfill-space/datafile 2020-09-12T14:36:28.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:28.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-224 td/osd-backfill-space/datafile 2020-09-12T14:36:28.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:28.744 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:28.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:28.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-225 td/osd-backfill-space/datafile 2020-09-12T14:36:28.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:28.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-225 td/osd-backfill-space/datafile 2020-09-12T14:36:28.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:28.962 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:28.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:28.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-226 td/osd-backfill-space/datafile 2020-09-12T14:36:29.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:29.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-226 td/osd-backfill-space/datafile 2020-09-12T14:36:29.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:29.171 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:29.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:29.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-227 td/osd-backfill-space/datafile 2020-09-12T14:36:29.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:29.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-227 td/osd-backfill-space/datafile 2020-09-12T14:36:29.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:29.388 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:29.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:29.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-228 td/osd-backfill-space/datafile 2020-09-12T14:36:29.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:29.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-228 td/osd-backfill-space/datafile 2020-09-12T14:36:29.622 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:29.623 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:29.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:29.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-229 td/osd-backfill-space/datafile 2020-09-12T14:36:29.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:29.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-229 td/osd-backfill-space/datafile 2020-09-12T14:36:29.848 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:29.849 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:29.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:29.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-230 td/osd-backfill-space/datafile 2020-09-12T14:36:29.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:29.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-230 td/osd-backfill-space/datafile 2020-09-12T14:36:30.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:30.050 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:30.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:30.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-231 td/osd-backfill-space/datafile 2020-09-12T14:36:30.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:30.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-231 td/osd-backfill-space/datafile 2020-09-12T14:36:30.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:30.259 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:30.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:30.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-232 td/osd-backfill-space/datafile 2020-09-12T14:36:30.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:30.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-232 td/osd-backfill-space/datafile 2020-09-12T14:36:30.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:30.485 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:30.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:30.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-233 td/osd-backfill-space/datafile 2020-09-12T14:36:30.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:30.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-233 td/osd-backfill-space/datafile 2020-09-12T14:36:30.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:30.736 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:30.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:30.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-234 td/osd-backfill-space/datafile 2020-09-12T14:36:30.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:30.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-234 td/osd-backfill-space/datafile 2020-09-12T14:36:31.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:31.013 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:31.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:31.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-235 td/osd-backfill-space/datafile 2020-09-12T14:36:31.113 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:31.113 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-235 td/osd-backfill-space/datafile 2020-09-12T14:36:31.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:31.214 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:31.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:31.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-236 td/osd-backfill-space/datafile 2020-09-12T14:36:31.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:31.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-236 td/osd-backfill-space/datafile 2020-09-12T14:36:31.439 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:31.440 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:31.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:31.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-237 td/osd-backfill-space/datafile 2020-09-12T14:36:31.548 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:31.548 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-237 td/osd-backfill-space/datafile 2020-09-12T14:36:31.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:31.657 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:31.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:31.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-238 td/osd-backfill-space/datafile 2020-09-12T14:36:31.765 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:31.766 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-238 td/osd-backfill-space/datafile 2020-09-12T14:36:31.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:31.875 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:31.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:31.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-239 td/osd-backfill-space/datafile 2020-09-12T14:36:31.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:31.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-239 td/osd-backfill-space/datafile 2020-09-12T14:36:32.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:32.101 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:32.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:32.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-240 td/osd-backfill-space/datafile 2020-09-12T14:36:32.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:32.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-240 td/osd-backfill-space/datafile 2020-09-12T14:36:32.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:32.327 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:32.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:32.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-241 td/osd-backfill-space/datafile 2020-09-12T14:36:32.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:32.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-241 td/osd-backfill-space/datafile 2020-09-12T14:36:32.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:32.578 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:32.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:32.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-242 td/osd-backfill-space/datafile 2020-09-12T14:36:32.728 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:32.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-242 td/osd-backfill-space/datafile 2020-09-12T14:36:32.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:32.838 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:32.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:32.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-243 td/osd-backfill-space/datafile 2020-09-12T14:36:32.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:32.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-243 td/osd-backfill-space/datafile 2020-09-12T14:36:33.038 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:33.039 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:33.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:33.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-244 td/osd-backfill-space/datafile 2020-09-12T14:36:33.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:33.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-244 td/osd-backfill-space/datafile 2020-09-12T14:36:33.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:33.248 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:33.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:33.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-245 td/osd-backfill-space/datafile 2020-09-12T14:36:33.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:33.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-245 td/osd-backfill-space/datafile 2020-09-12T14:36:33.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:33.499 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:33.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:33.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-246 td/osd-backfill-space/datafile 2020-09-12T14:36:33.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:33.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-246 td/osd-backfill-space/datafile 2020-09-12T14:36:33.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:33.717 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:33.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:33.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-247 td/osd-backfill-space/datafile 2020-09-12T14:36:33.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:33.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-247 td/osd-backfill-space/datafile 2020-09-12T14:36:33.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:33.959 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:33.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:33.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-248 td/osd-backfill-space/datafile 2020-09-12T14:36:34.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:34.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-248 td/osd-backfill-space/datafile 2020-09-12T14:36:34.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:34.194 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:34.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:34.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-249 td/osd-backfill-space/datafile 2020-09-12T14:36:34.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:34.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-249 td/osd-backfill-space/datafile 2020-09-12T14:36:34.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:34.462 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:34.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:34.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-250 td/osd-backfill-space/datafile 2020-09-12T14:36:34.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:34.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-250 td/osd-backfill-space/datafile 2020-09-12T14:36:34.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:34.714 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:34.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:34.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-251 td/osd-backfill-space/datafile 2020-09-12T14:36:34.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:34.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-251 td/osd-backfill-space/datafile 2020-09-12T14:36:34.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:34.948 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:34.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:34.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-252 td/osd-backfill-space/datafile 2020-09-12T14:36:35.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:35.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-252 td/osd-backfill-space/datafile 2020-09-12T14:36:35.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:35.182 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:35.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:35.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-253 td/osd-backfill-space/datafile 2020-09-12T14:36:35.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:35.291 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-253 td/osd-backfill-space/datafile 2020-09-12T14:36:35.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:35.408 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:35.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:35.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-254 td/osd-backfill-space/datafile 2020-09-12T14:36:35.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:35.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-254 td/osd-backfill-space/datafile 2020-09-12T14:36:35.625 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:35.626 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:35.626 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:35.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-255 td/osd-backfill-space/datafile 2020-09-12T14:36:35.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:35.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-255 td/osd-backfill-space/datafile 2020-09-12T14:36:35.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:35.877 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:35.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:35.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-256 td/osd-backfill-space/datafile 2020-09-12T14:36:35.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:35.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-256 td/osd-backfill-space/datafile 2020-09-12T14:36:36.094 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:36.095 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:36.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:36.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-257 td/osd-backfill-space/datafile 2020-09-12T14:36:36.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:36.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-257 td/osd-backfill-space/datafile 2020-09-12T14:36:36.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:36.304 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:36.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:36.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-258 td/osd-backfill-space/datafile 2020-09-12T14:36:36.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:36.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-258 td/osd-backfill-space/datafile 2020-09-12T14:36:36.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:36.522 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:36.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:36.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-259 td/osd-backfill-space/datafile 2020-09-12T14:36:36.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:36.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-259 td/osd-backfill-space/datafile 2020-09-12T14:36:36.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:36.748 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:36.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:36.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-260 td/osd-backfill-space/datafile 2020-09-12T14:36:36.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:36.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-260 td/osd-backfill-space/datafile 2020-09-12T14:36:36.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:36.991 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:36.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:36.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-261 td/osd-backfill-space/datafile 2020-09-12T14:36:37.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:37.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-261 td/osd-backfill-space/datafile 2020-09-12T14:36:37.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:37.242 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:37.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:37.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-262 td/osd-backfill-space/datafile 2020-09-12T14:36:37.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:37.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-262 td/osd-backfill-space/datafile 2020-09-12T14:36:37.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:37.493 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:37.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:37.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-263 td/osd-backfill-space/datafile 2020-09-12T14:36:37.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:37.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-263 td/osd-backfill-space/datafile 2020-09-12T14:36:37.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:37.745 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:37.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:37.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-264 td/osd-backfill-space/datafile 2020-09-12T14:36:37.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:37.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-264 td/osd-backfill-space/datafile 2020-09-12T14:36:37.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:37.996 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:37.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:37.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-265 td/osd-backfill-space/datafile 2020-09-12T14:36:38.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:38.113 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-265 td/osd-backfill-space/datafile 2020-09-12T14:36:38.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:38.230 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:38.231 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:38.231 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-266 td/osd-backfill-space/datafile 2020-09-12T14:36:38.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:38.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-266 td/osd-backfill-space/datafile 2020-09-12T14:36:38.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:38.541 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:38.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:38.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-267 td/osd-backfill-space/datafile 2020-09-12T14:36:38.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:38.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-267 td/osd-backfill-space/datafile 2020-09-12T14:36:38.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:38.816 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:38.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:38.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-268 td/osd-backfill-space/datafile 2020-09-12T14:36:38.933 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:38.933 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-268 td/osd-backfill-space/datafile 2020-09-12T14:36:39.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:39.051 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:39.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:39.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-269 td/osd-backfill-space/datafile 2020-09-12T14:36:39.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:39.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-269 td/osd-backfill-space/datafile 2020-09-12T14:36:39.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:39.302 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:39.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:39.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-270 td/osd-backfill-space/datafile 2020-09-12T14:36:39.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:39.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-270 td/osd-backfill-space/datafile 2020-09-12T14:36:39.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:39.553 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:39.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:39.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-271 td/osd-backfill-space/datafile 2020-09-12T14:36:39.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:39.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-271 td/osd-backfill-space/datafile 2020-09-12T14:36:39.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:39.813 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:39.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:39.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-272 td/osd-backfill-space/datafile 2020-09-12T14:36:39.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:39.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-272 td/osd-backfill-space/datafile 2020-09-12T14:36:40.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:40.064 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:40.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:40.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-273 td/osd-backfill-space/datafile 2020-09-12T14:36:40.197 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:40.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-273 td/osd-backfill-space/datafile 2020-09-12T14:36:40.315 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:40.315 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:40.316 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:40.316 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-274 td/osd-backfill-space/datafile 2020-09-12T14:36:40.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:40.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-274 td/osd-backfill-space/datafile 2020-09-12T14:36:40.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:40.591 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:40.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:40.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-275 td/osd-backfill-space/datafile 2020-09-12T14:36:40.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:40.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-275 td/osd-backfill-space/datafile 2020-09-12T14:36:40.859 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:40.859 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:40.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:40.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-276 td/osd-backfill-space/datafile 2020-09-12T14:36:40.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:40.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-276 td/osd-backfill-space/datafile 2020-09-12T14:36:41.110 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:41.111 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:41.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:41.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-277 td/osd-backfill-space/datafile 2020-09-12T14:36:41.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:41.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-277 td/osd-backfill-space/datafile 2020-09-12T14:36:41.362 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:41.363 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:41.363 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:41.363 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-278 td/osd-backfill-space/datafile 2020-09-12T14:36:41.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:41.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-278 td/osd-backfill-space/datafile 2020-09-12T14:36:41.604 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:41.605 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:41.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:41.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-279 td/osd-backfill-space/datafile 2020-09-12T14:36:41.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:41.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-279 td/osd-backfill-space/datafile 2020-09-12T14:36:41.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:41.889 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:41.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:41.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-280 td/osd-backfill-space/datafile 2020-09-12T14:36:42.014 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:42.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-280 td/osd-backfill-space/datafile 2020-09-12T14:36:42.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:42.133 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:42.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:42.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-281 td/osd-backfill-space/datafile 2020-09-12T14:36:42.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:42.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-281 td/osd-backfill-space/datafile 2020-09-12T14:36:42.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:42.375 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:42.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:42.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-282 td/osd-backfill-space/datafile 2020-09-12T14:36:42.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:42.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-282 td/osd-backfill-space/datafile 2020-09-12T14:36:42.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:42.601 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:42.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:42.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-283 td/osd-backfill-space/datafile 2020-09-12T14:36:42.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:42.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-283 td/osd-backfill-space/datafile 2020-09-12T14:36:42.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:42.862 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:42.862 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:42.862 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-284 td/osd-backfill-space/datafile 2020-09-12T14:36:42.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:42.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-284 td/osd-backfill-space/datafile 2020-09-12T14:36:43.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:43.096 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:43.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:43.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-285 td/osd-backfill-space/datafile 2020-09-12T14:36:43.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:43.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-285 td/osd-backfill-space/datafile 2020-09-12T14:36:43.371 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:43.372 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:43.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:43.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-286 td/osd-backfill-space/datafile 2020-09-12T14:36:43.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:43.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-286 td/osd-backfill-space/datafile 2020-09-12T14:36:43.823 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:43.824 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:43.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:43.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-287 td/osd-backfill-space/datafile 2020-09-12T14:36:43.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:43.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-287 td/osd-backfill-space/datafile 2020-09-12T14:36:44.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:44.075 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:44.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:44.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-288 td/osd-backfill-space/datafile 2020-09-12T14:36:44.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:44.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-288 td/osd-backfill-space/datafile 2020-09-12T14:36:44.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:44.326 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:44.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:44.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-289 td/osd-backfill-space/datafile 2020-09-12T14:36:44.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:44.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-289 td/osd-backfill-space/datafile 2020-09-12T14:36:44.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:44.594 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:44.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:44.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-290 td/osd-backfill-space/datafile 2020-09-12T14:36:44.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:44.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-290 td/osd-backfill-space/datafile 2020-09-12T14:36:44.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:44.879 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:44.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:44.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-291 td/osd-backfill-space/datafile 2020-09-12T14:36:45.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:45.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-291 td/osd-backfill-space/datafile 2020-09-12T14:36:45.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:45.139 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:45.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:45.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-292 td/osd-backfill-space/datafile 2020-09-12T14:36:45.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:45.264 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-292 td/osd-backfill-space/datafile 2020-09-12T14:36:45.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:45.390 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:45.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:45.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-293 td/osd-backfill-space/datafile 2020-09-12T14:36:45.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:45.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-293 td/osd-backfill-space/datafile 2020-09-12T14:36:45.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:45.658 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:45.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:45.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-294 td/osd-backfill-space/datafile 2020-09-12T14:36:45.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:45.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-294 td/osd-backfill-space/datafile 2020-09-12T14:36:45.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:45.926 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:45.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:45.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-295 td/osd-backfill-space/datafile 2020-09-12T14:36:46.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:46.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-295 td/osd-backfill-space/datafile 2020-09-12T14:36:46.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:46.185 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:46.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:46.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-296 td/osd-backfill-space/datafile 2020-09-12T14:36:46.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:46.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-296 td/osd-backfill-space/datafile 2020-09-12T14:36:46.470 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:46.470 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:46.470 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:46.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-297 td/osd-backfill-space/datafile 2020-09-12T14:36:46.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:46.604 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-297 td/osd-backfill-space/datafile 2020-09-12T14:36:46.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:46.730 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:46.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:46.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-298 td/osd-backfill-space/datafile 2020-09-12T14:36:46.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:46.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-298 td/osd-backfill-space/datafile 2020-09-12T14:36:46.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:46.981 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:46.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:46.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-299 td/osd-backfill-space/datafile 2020-09-12T14:36:47.106 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:47.106 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-299 td/osd-backfill-space/datafile 2020-09-12T14:36:47.240 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:47.240 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:47.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:47.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-300 td/osd-backfill-space/datafile 2020-09-12T14:36:47.365 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:47.366 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-300 td/osd-backfill-space/datafile 2020-09-12T14:36:47.508 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:47.508 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:47.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:47.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-301 td/osd-backfill-space/datafile 2020-09-12T14:36:47.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:47.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-301 td/osd-backfill-space/datafile 2020-09-12T14:36:47.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:47.776 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:47.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:47.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-302 td/osd-backfill-space/datafile 2020-09-12T14:36:47.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:47.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-302 td/osd-backfill-space/datafile 2020-09-12T14:36:48.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:48.027 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:48.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:48.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-303 td/osd-backfill-space/datafile 2020-09-12T14:36:48.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:48.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-303 td/osd-backfill-space/datafile 2020-09-12T14:36:48.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:48.295 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:48.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:48.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-304 td/osd-backfill-space/datafile 2020-09-12T14:36:48.438 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:48.438 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-304 td/osd-backfill-space/datafile 2020-09-12T14:36:48.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:48.563 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:48.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:48.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-305 td/osd-backfill-space/datafile 2020-09-12T14:36:48.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:48.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-305 td/osd-backfill-space/datafile 2020-09-12T14:36:48.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:48.823 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:48.823 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:48.823 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-306 td/osd-backfill-space/datafile 2020-09-12T14:36:48.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:48.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-306 td/osd-backfill-space/datafile 2020-09-12T14:36:49.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:49.074 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:49.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:49.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-307 td/osd-backfill-space/datafile 2020-09-12T14:36:49.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:49.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-307 td/osd-backfill-space/datafile 2020-09-12T14:36:49.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:49.392 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:49.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:49.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-308 td/osd-backfill-space/datafile 2020-09-12T14:36:49.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:49.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-308 td/osd-backfill-space/datafile 2020-09-12T14:36:49.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:49.643 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:49.644 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:49.644 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-309 td/osd-backfill-space/datafile 2020-09-12T14:36:49.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:49.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-309 td/osd-backfill-space/datafile 2020-09-12T14:36:49.894 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:49.895 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:49.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:49.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-310 td/osd-backfill-space/datafile 2020-09-12T14:36:50.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:50.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-310 td/osd-backfill-space/datafile 2020-09-12T14:36:50.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:50.162 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:50.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:50.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-311 td/osd-backfill-space/datafile 2020-09-12T14:36:50.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:50.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-311 td/osd-backfill-space/datafile 2020-09-12T14:36:50.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:50.431 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:50.431 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:50.431 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-312 td/osd-backfill-space/datafile 2020-09-12T14:36:50.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:50.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-312 td/osd-backfill-space/datafile 2020-09-12T14:36:50.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:50.698 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:50.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:50.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-313 td/osd-backfill-space/datafile 2020-09-12T14:36:50.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:50.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-313 td/osd-backfill-space/datafile 2020-09-12T14:36:50.966 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:50.966 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:50.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:50.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-314 td/osd-backfill-space/datafile 2020-09-12T14:36:51.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:51.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-314 td/osd-backfill-space/datafile 2020-09-12T14:36:51.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:51.243 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:51.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:51.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-315 td/osd-backfill-space/datafile 2020-09-12T14:36:51.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:51.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-315 td/osd-backfill-space/datafile 2020-09-12T14:36:51.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:51.494 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:51.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:51.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-316 td/osd-backfill-space/datafile 2020-09-12T14:36:51.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:51.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-316 td/osd-backfill-space/datafile 2020-09-12T14:36:51.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:51.741 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:51.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:51.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-317 td/osd-backfill-space/datafile 2020-09-12T14:36:51.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:51.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-317 td/osd-backfill-space/datafile 2020-09-12T14:36:51.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:51.992 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:51.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:51.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-318 td/osd-backfill-space/datafile 2020-09-12T14:36:52.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:52.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-318 td/osd-backfill-space/datafile 2020-09-12T14:36:52.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:52.251 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:52.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:52.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-319 td/osd-backfill-space/datafile 2020-09-12T14:36:52.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:52.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-319 td/osd-backfill-space/datafile 2020-09-12T14:36:52.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:52.528 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:52.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:52.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-320 td/osd-backfill-space/datafile 2020-09-12T14:36:52.661 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:52.662 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-320 td/osd-backfill-space/datafile 2020-09-12T14:36:52.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:52.796 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:52.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:52.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-321 td/osd-backfill-space/datafile 2020-09-12T14:36:52.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:52.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-321 td/osd-backfill-space/datafile 2020-09-12T14:36:53.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:53.056 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:53.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:53.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-322 td/osd-backfill-space/datafile 2020-09-12T14:36:53.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:53.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-322 td/osd-backfill-space/datafile 2020-09-12T14:36:53.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:53.315 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:53.316 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:53.316 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-323 td/osd-backfill-space/datafile 2020-09-12T14:36:53.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:53.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-323 td/osd-backfill-space/datafile 2020-09-12T14:36:53.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:53.566 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:53.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:53.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-324 td/osd-backfill-space/datafile 2020-09-12T14:36:53.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:53.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-324 td/osd-backfill-space/datafile 2020-09-12T14:36:53.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:53.826 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:53.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:53.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-325 td/osd-backfill-space/datafile 2020-09-12T14:36:53.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:53.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-325 td/osd-backfill-space/datafile 2020-09-12T14:36:54.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:54.060 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:54.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:54.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-326 td/osd-backfill-space/datafile 2020-09-12T14:36:54.177 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:54.177 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-326 td/osd-backfill-space/datafile 2020-09-12T14:36:54.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:54.303 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:54.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:54.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-327 td/osd-backfill-space/datafile 2020-09-12T14:36:54.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:54.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-327 td/osd-backfill-space/datafile 2020-09-12T14:36:54.612 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:54.613 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:54.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:54.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-328 td/osd-backfill-space/datafile 2020-09-12T14:36:54.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:54.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-328 td/osd-backfill-space/datafile 2020-09-12T14:36:54.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:54.856 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:54.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:54.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-329 td/osd-backfill-space/datafile 2020-09-12T14:36:54.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:54.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-329 td/osd-backfill-space/datafile 2020-09-12T14:36:55.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:55.082 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:55.083 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:55.083 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-330 td/osd-backfill-space/datafile 2020-09-12T14:36:55.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:55.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-330 td/osd-backfill-space/datafile 2020-09-12T14:36:55.316 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:55.317 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:55.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:55.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-331 td/osd-backfill-space/datafile 2020-09-12T14:36:55.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:55.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-331 td/osd-backfill-space/datafile 2020-09-12T14:36:55.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:55.589 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:55.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:55.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-332 td/osd-backfill-space/datafile 2020-09-12T14:36:55.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:55.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-332 td/osd-backfill-space/datafile 2020-09-12T14:36:55.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:55.844 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:55.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:55.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-333 td/osd-backfill-space/datafile 2020-09-12T14:36:55.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:55.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-333 td/osd-backfill-space/datafile 2020-09-12T14:36:56.079 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:56.080 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:56.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:56.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-334 td/osd-backfill-space/datafile 2020-09-12T14:36:56.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:56.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-334 td/osd-backfill-space/datafile 2020-09-12T14:36:56.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:56.330 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:56.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:56.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-335 td/osd-backfill-space/datafile 2020-09-12T14:36:56.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:56.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-335 td/osd-backfill-space/datafile 2020-09-12T14:36:56.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:56.573 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:56.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:56.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-336 td/osd-backfill-space/datafile 2020-09-12T14:36:56.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:56.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-336 td/osd-backfill-space/datafile 2020-09-12T14:36:56.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:56.825 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:56.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:56.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-337 td/osd-backfill-space/datafile 2020-09-12T14:36:56.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:56.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-337 td/osd-backfill-space/datafile 2020-09-12T14:36:57.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:57.068 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:57.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:57.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-338 td/osd-backfill-space/datafile 2020-09-12T14:36:57.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:57.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-338 td/osd-backfill-space/datafile 2020-09-12T14:36:57.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:57.310 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:57.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:57.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-339 td/osd-backfill-space/datafile 2020-09-12T14:36:57.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:57.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-339 td/osd-backfill-space/datafile 2020-09-12T14:36:57.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:57.545 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:57.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:57.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-340 td/osd-backfill-space/datafile 2020-09-12T14:36:57.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:57.671 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-340 td/osd-backfill-space/datafile 2020-09-12T14:36:57.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:57.796 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:57.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:57.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-341 td/osd-backfill-space/datafile 2020-09-12T14:36:57.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:57.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-341 td/osd-backfill-space/datafile 2020-09-12T14:36:58.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:58.047 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:58.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:58.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-342 td/osd-backfill-space/datafile 2020-09-12T14:36:58.164 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:58.164 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-342 td/osd-backfill-space/datafile 2020-09-12T14:36:58.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:58.290 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:58.291 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:58.291 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-343 td/osd-backfill-space/datafile 2020-09-12T14:36:58.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:58.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-343 td/osd-backfill-space/datafile 2020-09-12T14:36:58.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:58.525 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:58.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:58.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-344 td/osd-backfill-space/datafile 2020-09-12T14:36:58.667 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:58.668 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-344 td/osd-backfill-space/datafile 2020-09-12T14:36:58.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:58.793 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:58.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:58.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-345 td/osd-backfill-space/datafile 2020-09-12T14:36:58.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:58.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-345 td/osd-backfill-space/datafile 2020-09-12T14:36:59.035 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:59.036 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:59.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:59.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-346 td/osd-backfill-space/datafile 2020-09-12T14:36:59.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:59.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-346 td/osd-backfill-space/datafile 2020-09-12T14:36:59.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:59.287 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:59.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:59.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-347 td/osd-backfill-space/datafile 2020-09-12T14:36:59.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:59.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-347 td/osd-backfill-space/datafile 2020-09-12T14:36:59.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:59.524 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:59.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:59.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-348 td/osd-backfill-space/datafile 2020-09-12T14:36:59.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:59.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-348 td/osd-backfill-space/datafile 2020-09-12T14:36:59.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:36:59.758 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:36:59.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:59.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-349 td/osd-backfill-space/datafile 2020-09-12T14:36:59.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:36:59.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-349 td/osd-backfill-space/datafile 2020-09-12T14:37:00.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:00.001 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:00.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:00.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-350 td/osd-backfill-space/datafile 2020-09-12T14:37:00.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:00.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-350 td/osd-backfill-space/datafile 2020-09-12T14:37:00.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:00.252 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:00.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:00.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-351 td/osd-backfill-space/datafile 2020-09-12T14:37:00.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:00.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-351 td/osd-backfill-space/datafile 2020-09-12T14:37:00.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:00.495 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:00.496 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:00.496 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-352 td/osd-backfill-space/datafile 2020-09-12T14:37:00.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:00.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-352 td/osd-backfill-space/datafile 2020-09-12T14:37:00.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:00.763 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:00.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:00.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-353 td/osd-backfill-space/datafile 2020-09-12T14:37:00.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:00.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-353 td/osd-backfill-space/datafile 2020-09-12T14:37:01.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:01.031 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:01.032 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:01.032 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-354 td/osd-backfill-space/datafile 2020-09-12T14:37:01.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:01.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-354 td/osd-backfill-space/datafile 2020-09-12T14:37:01.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:01.274 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:01.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:01.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-355 td/osd-backfill-space/datafile 2020-09-12T14:37:01.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:01.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-355 td/osd-backfill-space/datafile 2020-09-12T14:37:01.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:01.517 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:01.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:01.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-356 td/osd-backfill-space/datafile 2020-09-12T14:37:01.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:01.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-356 td/osd-backfill-space/datafile 2020-09-12T14:37:01.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:01.751 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:01.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:01.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-357 td/osd-backfill-space/datafile 2020-09-12T14:37:01.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:01.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-357 td/osd-backfill-space/datafile 2020-09-12T14:37:02.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:02.003 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:02.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:02.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-358 td/osd-backfill-space/datafile 2020-09-12T14:37:02.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:02.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-358 td/osd-backfill-space/datafile 2020-09-12T14:37:02.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:02.237 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:02.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:02.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-359 td/osd-backfill-space/datafile 2020-09-12T14:37:02.354 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:02.354 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-359 td/osd-backfill-space/datafile 2020-09-12T14:37:02.480 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:02.480 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:02.481 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:02.481 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-360 td/osd-backfill-space/datafile 2020-09-12T14:37:02.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:02.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-360 td/osd-backfill-space/datafile 2020-09-12T14:37:02.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:02.740 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:02.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:02.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-361 td/osd-backfill-space/datafile 2020-09-12T14:37:02.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:02.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-361 td/osd-backfill-space/datafile 2020-09-12T14:37:02.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:02.992 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:02.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:02.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-362 td/osd-backfill-space/datafile 2020-09-12T14:37:03.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:03.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-362 td/osd-backfill-space/datafile 2020-09-12T14:37:03.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:03.227 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:03.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:03.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-363 td/osd-backfill-space/datafile 2020-09-12T14:37:03.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:03.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-363 td/osd-backfill-space/datafile 2020-09-12T14:37:03.468 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:03.469 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:03.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:03.470 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-364 td/osd-backfill-space/datafile 2020-09-12T14:37:03.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:03.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-364 td/osd-backfill-space/datafile 2020-09-12T14:37:03.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:03.712 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:03.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:03.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-365 td/osd-backfill-space/datafile 2020-09-12T14:37:03.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:03.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-365 td/osd-backfill-space/datafile 2020-09-12T14:37:03.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:03.947 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:03.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:03.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-366 td/osd-backfill-space/datafile 2020-09-12T14:37:04.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:04.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-366 td/osd-backfill-space/datafile 2020-09-12T14:37:04.197 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:04.198 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:04.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:04.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-367 td/osd-backfill-space/datafile 2020-09-12T14:37:04.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:04.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-367 td/osd-backfill-space/datafile 2020-09-12T14:37:04.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:04.466 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:04.466 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:04.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-368 td/osd-backfill-space/datafile 2020-09-12T14:37:04.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:04.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-368 td/osd-backfill-space/datafile 2020-09-12T14:37:04.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:04.809 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:04.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:04.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-369 td/osd-backfill-space/datafile 2020-09-12T14:37:04.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:04.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-369 td/osd-backfill-space/datafile 2020-09-12T14:37:05.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:05.043 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:05.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:05.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-370 td/osd-backfill-space/datafile 2020-09-12T14:37:05.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:05.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-370 td/osd-backfill-space/datafile 2020-09-12T14:37:05.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:05.294 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:05.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:05.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-371 td/osd-backfill-space/datafile 2020-09-12T14:37:05.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:05.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-371 td/osd-backfill-space/datafile 2020-09-12T14:37:05.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:05.546 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:05.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:05.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-372 td/osd-backfill-space/datafile 2020-09-12T14:37:05.662 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:05.662 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-372 td/osd-backfill-space/datafile 2020-09-12T14:37:05.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:05.814 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:05.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:05.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-373 td/osd-backfill-space/datafile 2020-09-12T14:37:05.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:05.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-373 td/osd-backfill-space/datafile 2020-09-12T14:37:06.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:06.073 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:06.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:06.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-374 td/osd-backfill-space/datafile 2020-09-12T14:37:06.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:06.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-374 td/osd-backfill-space/datafile 2020-09-12T14:37:06.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:06.333 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:06.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:06.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-375 td/osd-backfill-space/datafile 2020-09-12T14:37:06.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:06.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-375 td/osd-backfill-space/datafile 2020-09-12T14:37:06.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:06.576 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:06.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:06.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-376 td/osd-backfill-space/datafile 2020-09-12T14:37:06.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:06.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-376 td/osd-backfill-space/datafile 2020-09-12T14:37:06.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:06.844 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:06.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:06.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-377 td/osd-backfill-space/datafile 2020-09-12T14:37:06.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:06.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-377 td/osd-backfill-space/datafile 2020-09-12T14:37:07.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:07.121 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:07.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:07.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-378 td/osd-backfill-space/datafile 2020-09-12T14:37:07.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:07.246 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-378 td/osd-backfill-space/datafile 2020-09-12T14:37:07.362 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:07.363 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:07.363 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:07.363 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-379 td/osd-backfill-space/datafile 2020-09-12T14:37:07.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:07.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-379 td/osd-backfill-space/datafile 2020-09-12T14:37:07.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:07.606 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:07.606 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:07.606 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-380 td/osd-backfill-space/datafile 2020-09-12T14:37:07.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:07.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-380 td/osd-backfill-space/datafile 2020-09-12T14:37:07.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:07.882 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:07.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:07.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-381 td/osd-backfill-space/datafile 2020-09-12T14:37:08.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:08.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-381 td/osd-backfill-space/datafile 2020-09-12T14:37:08.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:08.134 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:08.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:08.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-382 td/osd-backfill-space/datafile 2020-09-12T14:37:08.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:08.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-382 td/osd-backfill-space/datafile 2020-09-12T14:37:08.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:08.402 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:08.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:08.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-383 td/osd-backfill-space/datafile 2020-09-12T14:37:08.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:08.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-383 td/osd-backfill-space/datafile 2020-09-12T14:37:08.669 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:08.670 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:08.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:08.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-384 td/osd-backfill-space/datafile 2020-09-12T14:37:08.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:08.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-384 td/osd-backfill-space/datafile 2020-09-12T14:37:08.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:08.937 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:08.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:08.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-385 td/osd-backfill-space/datafile 2020-09-12T14:37:09.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:09.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-385 td/osd-backfill-space/datafile 2020-09-12T14:37:09.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:09.189 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:09.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:09.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-386 td/osd-backfill-space/datafile 2020-09-12T14:37:09.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:09.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-386 td/osd-backfill-space/datafile 2020-09-12T14:37:09.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:09.448 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:09.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:09.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-387 td/osd-backfill-space/datafile 2020-09-12T14:37:09.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:09.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-387 td/osd-backfill-space/datafile 2020-09-12T14:37:09.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:09.733 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:09.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:09.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-388 td/osd-backfill-space/datafile 2020-09-12T14:37:09.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:09.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-388 td/osd-backfill-space/datafile 2020-09-12T14:37:09.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:09.975 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:09.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:09.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-389 td/osd-backfill-space/datafile 2020-09-12T14:37:10.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:10.093 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-389 td/osd-backfill-space/datafile 2020-09-12T14:37:10.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:10.210 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:10.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:10.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-390 td/osd-backfill-space/datafile 2020-09-12T14:37:10.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:10.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-390 td/osd-backfill-space/datafile 2020-09-12T14:37:10.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:10.486 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:10.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:10.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-391 td/osd-backfill-space/datafile 2020-09-12T14:37:10.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:10.612 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-391 td/osd-backfill-space/datafile 2020-09-12T14:37:10.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:10.730 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:10.730 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:10.730 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-392 td/osd-backfill-space/datafile 2020-09-12T14:37:10.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:10.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-392 td/osd-backfill-space/datafile 2020-09-12T14:37:10.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:10.972 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:10.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:10.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-393 td/osd-backfill-space/datafile 2020-09-12T14:37:11.106 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:11.106 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-393 td/osd-backfill-space/datafile 2020-09-12T14:37:11.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:11.224 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:11.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:11.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-394 td/osd-backfill-space/datafile 2020-09-12T14:37:11.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:11.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-394 td/osd-backfill-space/datafile 2020-09-12T14:37:11.466 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:11.467 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:11.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:11.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-395 td/osd-backfill-space/datafile 2020-09-12T14:37:11.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:11.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-395 td/osd-backfill-space/datafile 2020-09-12T14:37:11.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:11.718 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:11.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:11.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-396 td/osd-backfill-space/datafile 2020-09-12T14:37:11.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:11.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-396 td/osd-backfill-space/datafile 2020-09-12T14:37:11.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:11.961 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:11.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:11.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-397 td/osd-backfill-space/datafile 2020-09-12T14:37:12.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:12.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-397 td/osd-backfill-space/datafile 2020-09-12T14:37:12.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:12.204 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:12.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:12.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-398 td/osd-backfill-space/datafile 2020-09-12T14:37:12.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:12.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-398 td/osd-backfill-space/datafile 2020-09-12T14:37:12.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:12.472 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:12.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:12.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-399 td/osd-backfill-space/datafile 2020-09-12T14:37:12.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:12.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-399 td/osd-backfill-space/datafile 2020-09-12T14:37:12.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:12.731 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:12.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:12.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-400 td/osd-backfill-space/datafile 2020-09-12T14:37:12.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:12.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-400 td/osd-backfill-space/datafile 2020-09-12T14:37:12.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:12.983 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:12.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:12.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-401 td/osd-backfill-space/datafile 2020-09-12T14:37:13.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:13.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-401 td/osd-backfill-space/datafile 2020-09-12T14:37:13.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:13.218 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:13.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:13.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-402 td/osd-backfill-space/datafile 2020-09-12T14:37:13.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:13.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-402 td/osd-backfill-space/datafile 2020-09-12T14:37:13.468 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:13.469 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:13.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:13.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-403 td/osd-backfill-space/datafile 2020-09-12T14:37:13.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:13.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-403 td/osd-backfill-space/datafile 2020-09-12T14:37:13.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:13.703 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:13.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:13.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-404 td/osd-backfill-space/datafile 2020-09-12T14:37:13.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:13.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-404 td/osd-backfill-space/datafile 2020-09-12T14:37:13.953 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:13.954 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:13.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:13.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-405 td/osd-backfill-space/datafile 2020-09-12T14:37:14.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:14.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-405 td/osd-backfill-space/datafile 2020-09-12T14:37:14.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:14.189 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:14.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:14.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-406 td/osd-backfill-space/datafile 2020-09-12T14:37:14.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:14.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-406 td/osd-backfill-space/datafile 2020-09-12T14:37:14.457 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:14.457 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:14.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:14.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-407 td/osd-backfill-space/datafile 2020-09-12T14:37:14.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:14.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-407 td/osd-backfill-space/datafile 2020-09-12T14:37:14.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:14.725 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:14.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:14.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-408 td/osd-backfill-space/datafile 2020-09-12T14:37:15.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:15.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-408 td/osd-backfill-space/datafile 2020-09-12T14:37:15.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:15.193 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:15.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:15.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-409 td/osd-backfill-space/datafile 2020-09-12T14:37:15.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:15.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-409 td/osd-backfill-space/datafile 2020-09-12T14:37:15.444 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:15.444 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:15.445 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:15.445 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-410 td/osd-backfill-space/datafile 2020-09-12T14:37:15.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:15.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-410 td/osd-backfill-space/datafile 2020-09-12T14:37:15.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:15.695 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:15.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:15.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-411 td/osd-backfill-space/datafile 2020-09-12T14:37:15.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:15.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-411 td/osd-backfill-space/datafile 2020-09-12T14:37:15.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:15.963 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:15.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:15.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-412 td/osd-backfill-space/datafile 2020-09-12T14:37:16.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:16.089 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-412 td/osd-backfill-space/datafile 2020-09-12T14:37:16.222 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:16.223 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:16.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:16.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-413 td/osd-backfill-space/datafile 2020-09-12T14:37:16.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:16.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-413 td/osd-backfill-space/datafile 2020-09-12T14:37:16.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:16.508 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:16.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:16.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-414 td/osd-backfill-space/datafile 2020-09-12T14:37:16.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:16.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-414 td/osd-backfill-space/datafile 2020-09-12T14:37:16.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:16.776 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:16.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:16.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-415 td/osd-backfill-space/datafile 2020-09-12T14:37:16.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:16.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-415 td/osd-backfill-space/datafile 2020-09-12T14:37:17.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:17.019 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:17.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:17.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-416 td/osd-backfill-space/datafile 2020-09-12T14:37:17.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:17.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-416 td/osd-backfill-space/datafile 2020-09-12T14:37:17.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:17.253 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:17.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:17.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-417 td/osd-backfill-space/datafile 2020-09-12T14:37:17.378 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:17.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-417 td/osd-backfill-space/datafile 2020-09-12T14:37:17.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:17.504 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:17.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:17.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-418 td/osd-backfill-space/datafile 2020-09-12T14:37:17.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:17.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-418 td/osd-backfill-space/datafile 2020-09-12T14:37:17.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:17.747 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:17.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:17.748 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-419 td/osd-backfill-space/datafile 2020-09-12T14:37:17.864 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:17.864 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-419 td/osd-backfill-space/datafile 2020-09-12T14:37:17.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:17.999 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:17.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:17.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-420 td/osd-backfill-space/datafile 2020-09-12T14:37:18.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:18.116 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-420 td/osd-backfill-space/datafile 2020-09-12T14:37:18.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:18.250 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:18.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:18.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-421 td/osd-backfill-space/datafile 2020-09-12T14:37:18.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:18.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-421 td/osd-backfill-space/datafile 2020-09-12T14:37:18.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:18.501 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:18.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:18.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-422 td/osd-backfill-space/datafile 2020-09-12T14:37:18.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:18.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-422 td/osd-backfill-space/datafile 2020-09-12T14:37:18.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:18.744 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:18.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:18.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-423 td/osd-backfill-space/datafile 2020-09-12T14:37:18.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:18.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-423 td/osd-backfill-space/datafile 2020-09-12T14:37:18.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:18.987 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:18.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:18.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-424 td/osd-backfill-space/datafile 2020-09-12T14:37:19.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:19.113 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-424 td/osd-backfill-space/datafile 2020-09-12T14:37:19.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:19.238 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:19.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:19.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-425 td/osd-backfill-space/datafile 2020-09-12T14:37:19.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:19.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-425 td/osd-backfill-space/datafile 2020-09-12T14:37:19.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:19.498 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:19.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:19.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-426 td/osd-backfill-space/datafile 2020-09-12T14:37:19.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:19.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-426 td/osd-backfill-space/datafile 2020-09-12T14:37:19.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:19.732 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:19.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:19.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-427 td/osd-backfill-space/datafile 2020-09-12T14:37:19.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:19.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-427 td/osd-backfill-space/datafile 2020-09-12T14:37:19.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:19.984 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:19.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:19.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-428 td/osd-backfill-space/datafile 2020-09-12T14:37:20.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:20.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-428 td/osd-backfill-space/datafile 2020-09-12T14:37:20.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:20.227 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:20.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:20.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-429 td/osd-backfill-space/datafile 2020-09-12T14:37:20.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:20.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-429 td/osd-backfill-space/datafile 2020-09-12T14:37:20.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:20.528 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:20.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:20.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-430 td/osd-backfill-space/datafile 2020-09-12T14:37:20.653 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:20.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-430 td/osd-backfill-space/datafile 2020-09-12T14:37:20.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:20.779 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:20.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:20.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-431 td/osd-backfill-space/datafile 2020-09-12T14:37:20.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:20.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-431 td/osd-backfill-space/datafile 2020-09-12T14:37:21.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:21.047 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:21.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:21.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-432 td/osd-backfill-space/datafile 2020-09-12T14:37:21.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:21.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-432 td/osd-backfill-space/datafile 2020-09-12T14:37:21.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:21.307 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:21.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:21.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-433 td/osd-backfill-space/datafile 2020-09-12T14:37:21.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:21.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-433 td/osd-backfill-space/datafile 2020-09-12T14:37:21.558 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:21.558 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:21.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:21.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-434 td/osd-backfill-space/datafile 2020-09-12T14:37:21.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:21.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-434 td/osd-backfill-space/datafile 2020-09-12T14:37:21.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:21.810 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:21.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:21.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-435 td/osd-backfill-space/datafile 2020-09-12T14:37:21.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:21.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-435 td/osd-backfill-space/datafile 2020-09-12T14:37:22.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:22.044 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:22.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:22.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-436 td/osd-backfill-space/datafile 2020-09-12T14:37:22.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:22.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-436 td/osd-backfill-space/datafile 2020-09-12T14:37:22.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:22.278 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:22.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:22.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-437 td/osd-backfill-space/datafile 2020-09-12T14:37:22.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:22.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-437 td/osd-backfill-space/datafile 2020-09-12T14:37:22.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:22.555 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:22.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:22.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-438 td/osd-backfill-space/datafile 2020-09-12T14:37:22.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:22.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-438 td/osd-backfill-space/datafile 2020-09-12T14:37:22.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:22.815 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:22.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:22.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-439 td/osd-backfill-space/datafile 2020-09-12T14:37:22.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:22.932 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-439 td/osd-backfill-space/datafile 2020-09-12T14:37:23.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:23.049 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:23.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:23.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-440 td/osd-backfill-space/datafile 2020-09-12T14:37:23.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:23.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-440 td/osd-backfill-space/datafile 2020-09-12T14:37:23.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:23.292 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:23.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:23.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-441 td/osd-backfill-space/datafile 2020-09-12T14:37:23.417 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:23.417 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-441 td/osd-backfill-space/datafile 2020-09-12T14:37:23.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:23.543 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:23.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:23.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-442 td/osd-backfill-space/datafile 2020-09-12T14:37:23.660 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:23.660 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-442 td/osd-backfill-space/datafile 2020-09-12T14:37:23.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:23.778 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:23.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:23.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-443 td/osd-backfill-space/datafile 2020-09-12T14:37:23.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:23.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-443 td/osd-backfill-space/datafile 2020-09-12T14:37:24.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:24.037 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:24.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:24.038 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-444 td/osd-backfill-space/datafile 2020-09-12T14:37:24.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:24.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-444 td/osd-backfill-space/datafile 2020-09-12T14:37:24.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:24.280 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:24.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:24.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-445 td/osd-backfill-space/datafile 2020-09-12T14:37:24.422 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:24.423 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-445 td/osd-backfill-space/datafile 2020-09-12T14:37:24.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:24.540 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:24.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:24.541 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-446 td/osd-backfill-space/datafile 2020-09-12T14:37:24.673 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:24.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-446 td/osd-backfill-space/datafile 2020-09-12T14:37:24.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:24.791 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:24.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:24.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-447 td/osd-backfill-space/datafile 2020-09-12T14:37:24.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:24.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-447 td/osd-backfill-space/datafile 2020-09-12T14:37:25.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:25.051 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:25.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:25.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-448 td/osd-backfill-space/datafile 2020-09-12T14:37:25.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:25.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-448 td/osd-backfill-space/datafile 2020-09-12T14:37:25.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:25.302 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:25.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:25.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-449 td/osd-backfill-space/datafile 2020-09-12T14:37:25.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:25.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-449 td/osd-backfill-space/datafile 2020-09-12T14:37:25.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:25.553 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:25.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:25.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-450 td/osd-backfill-space/datafile 2020-09-12T14:37:25.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:25.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-450 td/osd-backfill-space/datafile 2020-09-12T14:37:25.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:25.788 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:25.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:25.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-451 td/osd-backfill-space/datafile 2020-09-12T14:37:25.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:25.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-451 td/osd-backfill-space/datafile 2020-09-12T14:37:26.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:26.023 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:26.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:26.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-452 td/osd-backfill-space/datafile 2020-09-12T14:37:26.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:26.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-452 td/osd-backfill-space/datafile 2020-09-12T14:37:26.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:26.291 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:26.291 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:26.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-453 td/osd-backfill-space/datafile 2020-09-12T14:37:26.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:26.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-453 td/osd-backfill-space/datafile 2020-09-12T14:37:26.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:26.533 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:26.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:26.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-454 td/osd-backfill-space/datafile 2020-09-12T14:37:26.659 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:26.659 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-454 td/osd-backfill-space/datafile 2020-09-12T14:37:26.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:26.785 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:26.785 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:26.785 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-455 td/osd-backfill-space/datafile 2020-09-12T14:37:26.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:26.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-455 td/osd-backfill-space/datafile 2020-09-12T14:37:27.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:27.044 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:27.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:27.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-456 td/osd-backfill-space/datafile 2020-09-12T14:37:27.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:27.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-456 td/osd-backfill-space/datafile 2020-09-12T14:37:27.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:27.279 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:27.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:27.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-457 td/osd-backfill-space/datafile 2020-09-12T14:37:27.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:27.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-457 td/osd-backfill-space/datafile 2020-09-12T14:37:27.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:27.513 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:27.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:27.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-458 td/osd-backfill-space/datafile 2020-09-12T14:37:27.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:27.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-458 td/osd-backfill-space/datafile 2020-09-12T14:37:27.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:27.756 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:27.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:27.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-459 td/osd-backfill-space/datafile 2020-09-12T14:37:27.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:27.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-459 td/osd-backfill-space/datafile 2020-09-12T14:37:28.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:28.024 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:28.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:28.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-460 td/osd-backfill-space/datafile 2020-09-12T14:37:28.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:28.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-460 td/osd-backfill-space/datafile 2020-09-12T14:37:28.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:28.259 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:28.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:28.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-461 td/osd-backfill-space/datafile 2020-09-12T14:37:28.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:28.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-461 td/osd-backfill-space/datafile 2020-09-12T14:37:28.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:28.518 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:28.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:28.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-462 td/osd-backfill-space/datafile 2020-09-12T14:37:28.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:28.644 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-462 td/osd-backfill-space/datafile 2020-09-12T14:37:28.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:28.770 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:28.771 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:28.771 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-463 td/osd-backfill-space/datafile 2020-09-12T14:37:28.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:28.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-463 td/osd-backfill-space/datafile 2020-09-12T14:37:29.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:29.021 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:29.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:29.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-464 td/osd-backfill-space/datafile 2020-09-12T14:37:29.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:29.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-464 td/osd-backfill-space/datafile 2020-09-12T14:37:29.264 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:29.264 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:29.264 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:29.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-465 td/osd-backfill-space/datafile 2020-09-12T14:37:29.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:29.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-465 td/osd-backfill-space/datafile 2020-09-12T14:37:29.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:29.515 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:29.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:29.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-466 td/osd-backfill-space/datafile 2020-09-12T14:37:29.649 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:29.649 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-466 td/osd-backfill-space/datafile 2020-09-12T14:37:29.766 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:29.767 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:29.767 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:29.767 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-467 td/osd-backfill-space/datafile 2020-09-12T14:37:29.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:29.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-467 td/osd-backfill-space/datafile 2020-09-12T14:37:30.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:30.017 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:30.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:30.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-468 td/osd-backfill-space/datafile 2020-09-12T14:37:30.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:30.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-468 td/osd-backfill-space/datafile 2020-09-12T14:37:30.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:30.269 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:30.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:30.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-469 td/osd-backfill-space/datafile 2020-09-12T14:37:30.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:30.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-469 td/osd-backfill-space/datafile 2020-09-12T14:37:30.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:30.520 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:30.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:30.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-470 td/osd-backfill-space/datafile 2020-09-12T14:37:30.662 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:30.663 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-470 td/osd-backfill-space/datafile 2020-09-12T14:37:30.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:30.780 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:30.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:30.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-471 td/osd-backfill-space/datafile 2020-09-12T14:37:30.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:30.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-471 td/osd-backfill-space/datafile 2020-09-12T14:37:31.014 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:31.014 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:31.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:31.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-472 td/osd-backfill-space/datafile 2020-09-12T14:37:31.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:31.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-472 td/osd-backfill-space/datafile 2020-09-12T14:37:31.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:31.300 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:31.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:31.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-473 td/osd-backfill-space/datafile 2020-09-12T14:37:31.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:31.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-473 td/osd-backfill-space/datafile 2020-09-12T14:37:31.558 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:31.559 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:31.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:31.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-474 td/osd-backfill-space/datafile 2020-09-12T14:37:31.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:31.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-474 td/osd-backfill-space/datafile 2020-09-12T14:37:31.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:31.793 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:31.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:31.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-475 td/osd-backfill-space/datafile 2020-09-12T14:37:31.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:31.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-475 td/osd-backfill-space/datafile 2020-09-12T14:37:32.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:32.044 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:32.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:32.045 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-476 td/osd-backfill-space/datafile 2020-09-12T14:37:32.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:32.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-476 td/osd-backfill-space/datafile 2020-09-12T14:37:32.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:32.279 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:32.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:32.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-477 td/osd-backfill-space/datafile 2020-09-12T14:37:32.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:32.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-477 td/osd-backfill-space/datafile 2020-09-12T14:37:32.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:32.530 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:32.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:32.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-478 td/osd-backfill-space/datafile 2020-09-12T14:37:32.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:32.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-478 td/osd-backfill-space/datafile 2020-09-12T14:37:32.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:32.773 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:32.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:32.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-479 td/osd-backfill-space/datafile 2020-09-12T14:37:32.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:32.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-479 td/osd-backfill-space/datafile 2020-09-12T14:37:33.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:33.016 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:33.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:33.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-480 td/osd-backfill-space/datafile 2020-09-12T14:37:33.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:33.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-480 td/osd-backfill-space/datafile 2020-09-12T14:37:33.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:33.267 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:33.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:33.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-481 td/osd-backfill-space/datafile 2020-09-12T14:37:33.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:33.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-481 td/osd-backfill-space/datafile 2020-09-12T14:37:33.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:33.502 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:33.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:33.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-482 td/osd-backfill-space/datafile 2020-09-12T14:37:33.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:33.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-482 td/osd-backfill-space/datafile 2020-09-12T14:37:33.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:33.745 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:33.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:33.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-483 td/osd-backfill-space/datafile 2020-09-12T14:37:33.862 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:33.862 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-483 td/osd-backfill-space/datafile 2020-09-12T14:37:33.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:33.979 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:33.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:33.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-484 td/osd-backfill-space/datafile 2020-09-12T14:37:34.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:34.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-484 td/osd-backfill-space/datafile 2020-09-12T14:37:34.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:34.214 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:34.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:34.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-485 td/osd-backfill-space/datafile 2020-09-12T14:37:34.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:34.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-485 td/osd-backfill-space/datafile 2020-09-12T14:37:34.482 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:34.482 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:34.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:34.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-486 td/osd-backfill-space/datafile 2020-09-12T14:37:34.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:34.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-486 td/osd-backfill-space/datafile 2020-09-12T14:37:34.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:34.735 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:34.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:34.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-487 td/osd-backfill-space/datafile 2020-09-12T14:37:34.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:34.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-487 td/osd-backfill-space/datafile 2020-09-12T14:37:34.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:34.968 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:34.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:34.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-488 td/osd-backfill-space/datafile 2020-09-12T14:37:35.085 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:35.085 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-488 td/osd-backfill-space/datafile 2020-09-12T14:37:35.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:35.211 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:35.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:35.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-489 td/osd-backfill-space/datafile 2020-09-12T14:37:35.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:35.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-489 td/osd-backfill-space/datafile 2020-09-12T14:37:35.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:35.479 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:35.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:35.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-490 td/osd-backfill-space/datafile 2020-09-12T14:37:35.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:35.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-490 td/osd-backfill-space/datafile 2020-09-12T14:37:35.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:35.713 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:35.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:35.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-491 td/osd-backfill-space/datafile 2020-09-12T14:37:35.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:35.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-491 td/osd-backfill-space/datafile 2020-09-12T14:37:35.997 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:35.998 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:35.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:35.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-492 td/osd-backfill-space/datafile 2020-09-12T14:37:36.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:36.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-492 td/osd-backfill-space/datafile 2020-09-12T14:37:36.240 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:36.241 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:36.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:36.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-493 td/osd-backfill-space/datafile 2020-09-12T14:37:36.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:36.371 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-493 td/osd-backfill-space/datafile 2020-09-12T14:37:36.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:36.484 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:36.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:36.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-494 td/osd-backfill-space/datafile 2020-09-12T14:37:36.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:36.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-494 td/osd-backfill-space/datafile 2020-09-12T14:37:36.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:36.744 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:36.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:36.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-495 td/osd-backfill-space/datafile 2020-09-12T14:37:36.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:36.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-495 td/osd-backfill-space/datafile 2020-09-12T14:37:36.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:36.978 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:36.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:36.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-496 td/osd-backfill-space/datafile 2020-09-12T14:37:37.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:37.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-496 td/osd-backfill-space/datafile 2020-09-12T14:37:37.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:37.213 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:37.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:37.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-497 td/osd-backfill-space/datafile 2020-09-12T14:37:37.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:37.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-497 td/osd-backfill-space/datafile 2020-09-12T14:37:37.447 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:37.447 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:37.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:37.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-498 td/osd-backfill-space/datafile 2020-09-12T14:37:37.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:37.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-498 td/osd-backfill-space/datafile 2020-09-12T14:37:37.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:37.715 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:37.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:37.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-499 td/osd-backfill-space/datafile 2020-09-12T14:37:37.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:37.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-499 td/osd-backfill-space/datafile 2020-09-12T14:37:37.966 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:37.967 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:37.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:37.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-500 td/osd-backfill-space/datafile 2020-09-12T14:37:38.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:38.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-500 td/osd-backfill-space/datafile 2020-09-12T14:37:38.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:38.209 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:38.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:38.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-501 td/osd-backfill-space/datafile 2020-09-12T14:37:38.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:38.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-501 td/osd-backfill-space/datafile 2020-09-12T14:37:38.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:38.452 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:38.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:38.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-502 td/osd-backfill-space/datafile 2020-09-12T14:37:38.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:38.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-502 td/osd-backfill-space/datafile 2020-09-12T14:37:38.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:38.695 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:38.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:38.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-503 td/osd-backfill-space/datafile 2020-09-12T14:37:38.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:38.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-503 td/osd-backfill-space/datafile 2020-09-12T14:37:38.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:38.955 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:38.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:38.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-504 td/osd-backfill-space/datafile 2020-09-12T14:37:39.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:39.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-504 td/osd-backfill-space/datafile 2020-09-12T14:37:39.197 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:39.197 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:39.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:39.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-505 td/osd-backfill-space/datafile 2020-09-12T14:37:39.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:39.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-505 td/osd-backfill-space/datafile 2020-09-12T14:37:39.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:39.449 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:39.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:39.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-506 td/osd-backfill-space/datafile 2020-09-12T14:37:39.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:39.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-506 td/osd-backfill-space/datafile 2020-09-12T14:37:39.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:39.717 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:39.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:39.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-507 td/osd-backfill-space/datafile 2020-09-12T14:37:39.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:39.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-507 td/osd-backfill-space/datafile 2020-09-12T14:37:39.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:39.951 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:39.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:39.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-508 td/osd-backfill-space/datafile 2020-09-12T14:37:40.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:40.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-508 td/osd-backfill-space/datafile 2020-09-12T14:37:40.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:40.186 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:40.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:40.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-509 td/osd-backfill-space/datafile 2020-09-12T14:37:40.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:40.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-509 td/osd-backfill-space/datafile 2020-09-12T14:37:40.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:40.429 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:40.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:40.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-510 td/osd-backfill-space/datafile 2020-09-12T14:37:40.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:40.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-510 td/osd-backfill-space/datafile 2020-09-12T14:37:40.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:40.714 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:40.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:40.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-511 td/osd-backfill-space/datafile 2020-09-12T14:37:40.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:40.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-511 td/osd-backfill-space/datafile 2020-09-12T14:37:40.997 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:40.998 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:40.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:40.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-512 td/osd-backfill-space/datafile 2020-09-12T14:37:41.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:41.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-512 td/osd-backfill-space/datafile 2020-09-12T14:37:41.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:41.233 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:41.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:41.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-513 td/osd-backfill-space/datafile 2020-09-12T14:37:41.342 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:41.342 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-513 td/osd-backfill-space/datafile 2020-09-12T14:37:41.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:41.467 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:41.468 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:41.468 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-514 td/osd-backfill-space/datafile 2020-09-12T14:37:41.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:41.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-514 td/osd-backfill-space/datafile 2020-09-12T14:37:41.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:41.710 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:41.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:41.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-515 td/osd-backfill-space/datafile 2020-09-12T14:37:41.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:41.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-515 td/osd-backfill-space/datafile 2020-09-12T14:37:41.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:41.970 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:41.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:41.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-516 td/osd-backfill-space/datafile 2020-09-12T14:37:42.086 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:42.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-516 td/osd-backfill-space/datafile 2020-09-12T14:37:42.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:42.204 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:42.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:42.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-517 td/osd-backfill-space/datafile 2020-09-12T14:37:42.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:42.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-517 td/osd-backfill-space/datafile 2020-09-12T14:37:42.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:42.473 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:42.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:42.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-518 td/osd-backfill-space/datafile 2020-09-12T14:37:42.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:42.590 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-518 td/osd-backfill-space/datafile 2020-09-12T14:37:42.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:42.724 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:42.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:42.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-519 td/osd-backfill-space/datafile 2020-09-12T14:37:42.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:42.841 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-519 td/osd-backfill-space/datafile 2020-09-12T14:37:42.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:42.958 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:42.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:42.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-520 td/osd-backfill-space/datafile 2020-09-12T14:37:43.083 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:43.084 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-520 td/osd-backfill-space/datafile 2020-09-12T14:37:43.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:43.209 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:43.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:43.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-521 td/osd-backfill-space/datafile 2020-09-12T14:37:43.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:43.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-521 td/osd-backfill-space/datafile 2020-09-12T14:37:43.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:43.461 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:43.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:43.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-522 td/osd-backfill-space/datafile 2020-09-12T14:37:43.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:43.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-522 td/osd-backfill-space/datafile 2020-09-12T14:37:43.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:43.712 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:43.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:43.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-523 td/osd-backfill-space/datafile 2020-09-12T14:37:43.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:43.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-523 td/osd-backfill-space/datafile 2020-09-12T14:37:43.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:43.955 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:43.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:43.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-524 td/osd-backfill-space/datafile 2020-09-12T14:37:44.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:44.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-524 td/osd-backfill-space/datafile 2020-09-12T14:37:44.197 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:44.197 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:44.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:44.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-525 td/osd-backfill-space/datafile 2020-09-12T14:37:44.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:44.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-525 td/osd-backfill-space/datafile 2020-09-12T14:37:44.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:44.449 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:44.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:44.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-526 td/osd-backfill-space/datafile 2020-09-12T14:37:44.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:44.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-526 td/osd-backfill-space/datafile 2020-09-12T14:37:44.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:44.692 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:44.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:44.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-527 td/osd-backfill-space/datafile 2020-09-12T14:37:44.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:44.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-527 td/osd-backfill-space/datafile 2020-09-12T14:37:44.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:44.960 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:44.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:44.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-528 td/osd-backfill-space/datafile 2020-09-12T14:37:45.085 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:45.085 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-528 td/osd-backfill-space/datafile 2020-09-12T14:37:45.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:45.211 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:45.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:45.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-529 td/osd-backfill-space/datafile 2020-09-12T14:37:45.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:45.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-529 td/osd-backfill-space/datafile 2020-09-12T14:37:45.445 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:45.445 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:45.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:45.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-530 td/osd-backfill-space/datafile 2020-09-12T14:37:45.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:45.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-530 td/osd-backfill-space/datafile 2020-09-12T14:37:45.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:45.697 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:45.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:45.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-531 td/osd-backfill-space/datafile 2020-09-12T14:37:45.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:45.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-531 td/osd-backfill-space/datafile 2020-09-12T14:37:45.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:45.939 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:45.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:45.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-532 td/osd-backfill-space/datafile 2020-09-12T14:37:46.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:46.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-532 td/osd-backfill-space/datafile 2020-09-12T14:37:46.433 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:46.433 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:46.434 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:46.434 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-533 td/osd-backfill-space/datafile 2020-09-12T14:37:46.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:46.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-533 td/osd-backfill-space/datafile 2020-09-12T14:37:46.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:46.701 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:46.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:46.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-534 td/osd-backfill-space/datafile 2020-09-12T14:37:46.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:46.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-534 td/osd-backfill-space/datafile 2020-09-12T14:37:46.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:46.977 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:46.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:46.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-535 td/osd-backfill-space/datafile 2020-09-12T14:37:47.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:47.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-535 td/osd-backfill-space/datafile 2020-09-12T14:37:47.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:47.245 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:47.246 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:47.246 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-536 td/osd-backfill-space/datafile 2020-09-12T14:37:47.378 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:47.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-536 td/osd-backfill-space/datafile 2020-09-12T14:37:47.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:47.504 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:47.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:47.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-537 td/osd-backfill-space/datafile 2020-09-12T14:37:47.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:47.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-537 td/osd-backfill-space/datafile 2020-09-12T14:37:47.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:47.764 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:47.765 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:47.765 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-538 td/osd-backfill-space/datafile 2020-09-12T14:37:47.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:47.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-538 td/osd-backfill-space/datafile 2020-09-12T14:37:48.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:48.041 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:48.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:48.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-539 td/osd-backfill-space/datafile 2020-09-12T14:37:48.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:48.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-539 td/osd-backfill-space/datafile 2020-09-12T14:37:48.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:48.309 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:48.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:48.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-540 td/osd-backfill-space/datafile 2020-09-12T14:37:48.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:48.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-540 td/osd-backfill-space/datafile 2020-09-12T14:37:48.584 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:48.585 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:48.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:48.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-541 td/osd-backfill-space/datafile 2020-09-12T14:37:48.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:48.728 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-541 td/osd-backfill-space/datafile 2020-09-12T14:37:48.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:48.869 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:48.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:48.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-542 td/osd-backfill-space/datafile 2020-09-12T14:37:49.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:49.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-542 td/osd-backfill-space/datafile 2020-09-12T14:37:49.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:49.137 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:49.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:49.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-543 td/osd-backfill-space/datafile 2020-09-12T14:37:49.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:49.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-543 td/osd-backfill-space/datafile 2020-09-12T14:37:49.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:49.405 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:49.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:49.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-544 td/osd-backfill-space/datafile 2020-09-12T14:37:49.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:49.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-544 td/osd-backfill-space/datafile 2020-09-12T14:37:49.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:49.681 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:49.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:49.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-545 td/osd-backfill-space/datafile 2020-09-12T14:37:49.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:49.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-545 td/osd-backfill-space/datafile 2020-09-12T14:37:49.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:49.949 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:49.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:49.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-546 td/osd-backfill-space/datafile 2020-09-12T14:37:50.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:50.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-546 td/osd-backfill-space/datafile 2020-09-12T14:37:50.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:50.209 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:50.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:50.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-547 td/osd-backfill-space/datafile 2020-09-12T14:37:50.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:50.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-547 td/osd-backfill-space/datafile 2020-09-12T14:37:50.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:50.477 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:50.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:50.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-548 td/osd-backfill-space/datafile 2020-09-12T14:37:50.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:50.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-548 td/osd-backfill-space/datafile 2020-09-12T14:37:50.744 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:50.745 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:50.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:50.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-549 td/osd-backfill-space/datafile 2020-09-12T14:37:50.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:50.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-549 td/osd-backfill-space/datafile 2020-09-12T14:37:51.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:51.021 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:51.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:51.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-550 td/osd-backfill-space/datafile 2020-09-12T14:37:51.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:51.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-550 td/osd-backfill-space/datafile 2020-09-12T14:37:51.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:51.289 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:51.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:51.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-551 td/osd-backfill-space/datafile 2020-09-12T14:37:51.423 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:51.423 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-551 td/osd-backfill-space/datafile 2020-09-12T14:37:51.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:51.557 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:51.557 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:51.558 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-552 td/osd-backfill-space/datafile 2020-09-12T14:37:51.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:51.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-552 td/osd-backfill-space/datafile 2020-09-12T14:37:51.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:51.902 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:51.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:51.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-553 td/osd-backfill-space/datafile 2020-09-12T14:37:52.038 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:52.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-553 td/osd-backfill-space/datafile 2020-09-12T14:37:52.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:52.162 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:52.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:52.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-554 td/osd-backfill-space/datafile 2020-09-12T14:37:52.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:52.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-554 td/osd-backfill-space/datafile 2020-09-12T14:37:52.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:52.421 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:52.422 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:52.422 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-555 td/osd-backfill-space/datafile 2020-09-12T14:37:52.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:52.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-555 td/osd-backfill-space/datafile 2020-09-12T14:37:52.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:52.698 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:52.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:52.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-556 td/osd-backfill-space/datafile 2020-09-12T14:37:52.823 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:52.823 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-556 td/osd-backfill-space/datafile 2020-09-12T14:37:52.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:52.949 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:52.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:52.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-557 td/osd-backfill-space/datafile 2020-09-12T14:37:53.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:53.083 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-557 td/osd-backfill-space/datafile 2020-09-12T14:37:53.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:53.208 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:53.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:53.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-558 td/osd-backfill-space/datafile 2020-09-12T14:37:53.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:53.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-558 td/osd-backfill-space/datafile 2020-09-12T14:37:53.468 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:53.468 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:53.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:53.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-559 td/osd-backfill-space/datafile 2020-09-12T14:37:53.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:53.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-559 td/osd-backfill-space/datafile 2020-09-12T14:37:53.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:53.728 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:53.728 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:53.728 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-560 td/osd-backfill-space/datafile 2020-09-12T14:37:53.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:53.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-560 td/osd-backfill-space/datafile 2020-09-12T14:37:53.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:53.979 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:53.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:53.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-561 td/osd-backfill-space/datafile 2020-09-12T14:37:54.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:54.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-561 td/osd-backfill-space/datafile 2020-09-12T14:37:54.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:54.214 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:54.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:54.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-562 td/osd-backfill-space/datafile 2020-09-12T14:37:54.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:54.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-562 td/osd-backfill-space/datafile 2020-09-12T14:37:54.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:54.490 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:54.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:54.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-563 td/osd-backfill-space/datafile 2020-09-12T14:37:54.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:54.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-563 td/osd-backfill-space/datafile 2020-09-12T14:37:54.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:54.758 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:54.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:54.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-564 td/osd-backfill-space/datafile 2020-09-12T14:37:54.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:54.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-564 td/osd-backfill-space/datafile 2020-09-12T14:37:55.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:55.026 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:55.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:55.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-565 td/osd-backfill-space/datafile 2020-09-12T14:37:55.151 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:55.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-565 td/osd-backfill-space/datafile 2020-09-12T14:37:55.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:55.277 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:55.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:55.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-566 td/osd-backfill-space/datafile 2020-09-12T14:37:55.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:55.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-566 td/osd-backfill-space/datafile 2020-09-12T14:37:55.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:55.520 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:55.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:55.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-567 td/osd-backfill-space/datafile 2020-09-12T14:37:55.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:55.654 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-567 td/osd-backfill-space/datafile 2020-09-12T14:37:55.771 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:55.772 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:55.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:55.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-568 td/osd-backfill-space/datafile 2020-09-12T14:37:55.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:55.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-568 td/osd-backfill-space/datafile 2020-09-12T14:37:56.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:56.006 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:56.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:56.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-569 td/osd-backfill-space/datafile 2020-09-12T14:37:56.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:56.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-569 td/osd-backfill-space/datafile 2020-09-12T14:37:56.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:56.249 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:56.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:56.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-570 td/osd-backfill-space/datafile 2020-09-12T14:37:56.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:56.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-570 td/osd-backfill-space/datafile 2020-09-12T14:37:56.508 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:56.509 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:56.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:56.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-571 td/osd-backfill-space/datafile 2020-09-12T14:37:56.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:56.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-571 td/osd-backfill-space/datafile 2020-09-12T14:37:56.785 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:56.786 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:56.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:56.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-572 td/osd-backfill-space/datafile 2020-09-12T14:37:56.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:56.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-572 td/osd-backfill-space/datafile 2020-09-12T14:37:57.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:57.045 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:57.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:57.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-573 td/osd-backfill-space/datafile 2020-09-12T14:37:57.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:57.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-573 td/osd-backfill-space/datafile 2020-09-12T14:37:57.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:57.296 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:57.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:57.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-574 td/osd-backfill-space/datafile 2020-09-12T14:37:57.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:57.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-574 td/osd-backfill-space/datafile 2020-09-12T14:37:57.547 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:57.547 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:57.548 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:57.548 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-575 td/osd-backfill-space/datafile 2020-09-12T14:37:57.664 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:57.664 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-575 td/osd-backfill-space/datafile 2020-09-12T14:37:57.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:57.782 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:57.782 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:57.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-576 td/osd-backfill-space/datafile 2020-09-12T14:37:57.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:57.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-576 td/osd-backfill-space/datafile 2020-09-12T14:37:58.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:58.016 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:58.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:58.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-577 td/osd-backfill-space/datafile 2020-09-12T14:37:58.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:58.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-577 td/osd-backfill-space/datafile 2020-09-12T14:37:58.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:58.268 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:58.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:58.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-578 td/osd-backfill-space/datafile 2020-09-12T14:37:58.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:58.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-578 td/osd-backfill-space/datafile 2020-09-12T14:37:58.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:58.519 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:58.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:58.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-579 td/osd-backfill-space/datafile 2020-09-12T14:37:58.661 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:58.661 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-579 td/osd-backfill-space/datafile 2020-09-12T14:37:58.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:58.779 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:58.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:58.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-580 td/osd-backfill-space/datafile 2020-09-12T14:37:58.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:58.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-580 td/osd-backfill-space/datafile 2020-09-12T14:37:59.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:59.030 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:59.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:59.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-581 td/osd-backfill-space/datafile 2020-09-12T14:37:59.164 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:59.164 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-581 td/osd-backfill-space/datafile 2020-09-12T14:37:59.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:59.290 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:59.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:59.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-582 td/osd-backfill-space/datafile 2020-09-12T14:37:59.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:59.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-582 td/osd-backfill-space/datafile 2020-09-12T14:37:59.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:59.532 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:59.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:59.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-583 td/osd-backfill-space/datafile 2020-09-12T14:37:59.649 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:59.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-583 td/osd-backfill-space/datafile 2020-09-12T14:37:59.766 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:37:59.767 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:37:59.767 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:59.767 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-584 td/osd-backfill-space/datafile 2020-09-12T14:37:59.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:37:59.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-584 td/osd-backfill-space/datafile 2020-09-12T14:38:00.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:38:00.027 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:38:00.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:00.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-585 td/osd-backfill-space/datafile 2020-09-12T14:38:00.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:00.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-585 td/osd-backfill-space/datafile 2020-09-12T14:38:00.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:38:00.278 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:38:00.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:00.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-586 td/osd-backfill-space/datafile 2020-09-12T14:38:00.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:00.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-586 td/osd-backfill-space/datafile 2020-09-12T14:38:00.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:38:00.512 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:38:00.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:00.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-587 td/osd-backfill-space/datafile 2020-09-12T14:38:00.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:00.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-587 td/osd-backfill-space/datafile 2020-09-12T14:38:00.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:38:00.772 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:38:00.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:00.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-588 td/osd-backfill-space/datafile 2020-09-12T14:38:00.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:00.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-588 td/osd-backfill-space/datafile 2020-09-12T14:38:01.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:38:01.023 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:38:01.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:01.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-589 td/osd-backfill-space/datafile 2020-09-12T14:38:01.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:01.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-589 td/osd-backfill-space/datafile 2020-09-12T14:38:01.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:38:01.258 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:38:01.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:01.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-590 td/osd-backfill-space/datafile 2020-09-12T14:38:01.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:01.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-590 td/osd-backfill-space/datafile 2020-09-12T14:38:01.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:38:01.527 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:38:01.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:01.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-591 td/osd-backfill-space/datafile 2020-09-12T14:38:01.660 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:01.660 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-591 td/osd-backfill-space/datafile 2020-09-12T14:38:01.785 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:38:01.786 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:38:01.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:01.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-592 td/osd-backfill-space/datafile 2020-09-12T14:38:01.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:01.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-592 td/osd-backfill-space/datafile 2020-09-12T14:38:02.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:38:02.028 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:38:02.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:02.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-593 td/osd-backfill-space/datafile 2020-09-12T14:38:02.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:02.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-593 td/osd-backfill-space/datafile 2020-09-12T14:38:02.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:38:02.271 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:38:02.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:02.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-594 td/osd-backfill-space/datafile 2020-09-12T14:38:02.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:02.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-594 td/osd-backfill-space/datafile 2020-09-12T14:38:02.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:38:02.523 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:38:02.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:02.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-595 td/osd-backfill-space/datafile 2020-09-12T14:38:02.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:02.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-595 td/osd-backfill-space/datafile 2020-09-12T14:38:02.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:38:02.791 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:38:02.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:02.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-596 td/osd-backfill-space/datafile 2020-09-12T14:38:02.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:02.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-596 td/osd-backfill-space/datafile 2020-09-12T14:38:03.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:38:03.059 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:38:03.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:03.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-597 td/osd-backfill-space/datafile 2020-09-12T14:38:03.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:03.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-597 td/osd-backfill-space/datafile 2020-09-12T14:38:03.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:38:03.310 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:38:03.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:03.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-598 td/osd-backfill-space/datafile 2020-09-12T14:38:03.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:03.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-598 td/osd-backfill-space/datafile 2020-09-12T14:38:03.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:38:03.570 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:38:03.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:03.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-599 td/osd-backfill-space/datafile 2020-09-12T14:38:03.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:03.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-599 td/osd-backfill-space/datafile 2020-09-12T14:38:03.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:500: TEST_backfill_multi_partial: for o in $(seq 1 $objects) 2020-09-12T14:38:03.821 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:38:03.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:03.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test1 put obj-1-600 td/osd-backfill-space/datafile 2020-09-12T14:38:03.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:502: TEST_backfill_multi_partial: for p in $(seq 1 $pools) 2020-09-12T14:38:03.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:504: TEST_backfill_multi_partial: rados -p test2 put obj-2-600 td/osd-backfill-space/datafile 2020-09-12T14:38:04.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:508: TEST_backfill_multi_partial: kill_daemon td/osd-backfill-space/osd.2.pid TERM 2020-09-12T14:38:04.055 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:255: kill_daemon: cat td/osd-backfill-space/osd.2.pid 2020-09-12T14:38:04.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:255: kill_daemon: local pid=207395 2020-09-12T14:38:04.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:256: kill_daemon: local send_signal=TERM 2020-09-12T14:38:04.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:257: kill_daemon: local 'delays=0.1 0.2 1 1 1 2 3 5 5 5 10 10 20 60 60 60 120' 2020-09-12T14:38:04.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:258: kill_daemon: local exit_code=1 2020-09-12T14:38:04.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:261: kill_daemon: for try in $delays 0 2020-09-12T14:38:04.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: kill -TERM 207395 2020-09-12T14:38:04.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:263: kill_daemon: exit_code=1 2020-09-12T14:38:04.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:268: kill_daemon: send_signal=0 2020-09-12T14:38:04.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:269: kill_daemon: sleep 0.1 2020-09-12T14:38:04.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:261: kill_daemon: for try in $delays 0 2020-09-12T14:38:04.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: kill -0 207395 2020-09-12T14:38:04.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:265: kill_daemon: exit_code=0 2020-09-12T14:38:04.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:266: kill_daemon: break 2020-09-12T14:38:04.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:271: kill_daemon: return 0 2020-09-12T14:38:04.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:509: TEST_backfill_multi_partial: ceph osd out osd.2 2020-09-12T14:38:04.939 INFO:tasks.workunit.client.0.smithi027.stderr:osd.2 is already out. 2020-09-12T14:38:04.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:511: TEST_backfill_multi_partial: activate_osd td/osd-backfill-space 1 2020-09-12T14:38:04.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: local dir=td/osd-backfill-space 2020-09-12T14:38:04.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: shift 2020-09-12T14:38:04.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local id=1 2020-09-12T14:38:04.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2020-09-12T14:38:04.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local osd_data=td/osd-backfill-space/1 2020-09-12T14:38:04.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:38:04.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:38:04.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:38:04.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:38:04.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-data=td/osd-backfill-space/1' 2020-09-12T14:38:04.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/1/journal' 2020-09-12T14:38:04.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --chdir=' 2020-09-12T14:38:04.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+= 2020-09-12T14:38:04.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:38:04.952 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: get_asok_path 2020-09-12T14:38:04.952 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:38:04.953 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:38:04.953 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:38:04.953 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:38:04.953 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:38:04.953 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:38:04.953 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:38:04.953 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:38:04.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:38:04.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:38:04.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:38:04.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:38:04.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:38:04.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' ' 2020-09-12T14:38:04.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+= 2020-09-12T14:38:04.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: mkdir -p td/osd-backfill-space/1 2020-09-12T14:38:04.955 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T14:38:04.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: echo start osd.1 2020-09-12T14:38:04.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: ceph-osd -i 1 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/1 --osd-journal=td/osd-backfill-space/1/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:38:04.956 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: cat td/osd-backfill-space/1/whoami 2020-09-12T14:38:04.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: '[' 1 = 1 ']' 2020-09-12T14:38:04.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: ceph osd dump --format=json 2020-09-12T14:38:04.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: jq '.flags_set[]' 2020-09-12T14:38:04.957 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: grep -q '"noup"' 2020-09-12T14:38:04.976 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:38:04.975+0000 7f6afe165ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:38:04.978 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:38:04.977+0000 7f6afe165ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:38:04.980 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:38:04.978+0000 7f6afe165ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:38:05.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: wait_for_osd up 1 2020-09-12T14:38:05.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:38:05.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:38:05.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:38:05.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:38:05.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:38:05.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:38:05.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:38:05.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:38:05.359 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:38:05.496 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:38:05.494+0000 7f6afe165ec0 -1 Falling back to public interface 2020-09-12T14:38:05.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:38:06.497 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:38:06.496+0000 7f6afe165ec0 -1 osd.1 43 log_to_monitors {default=true} 2020-09-12T14:38:06.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:38:06.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:38:06.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:38:06.787 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:38:06.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:38:06.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:38:07.251 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up out weight 0 up_from 48 up_thru 49 down_at 44 last_clean_interval [39,43) [v2:127.0.0.1:6810/233306,v1:127.0.0.1:6811/233306] [v2:127.0.0.1:6812/233306,v1:127.0.0.1:6813/233306] exists,up ce656e6a-6430-4cc2-bd95-29130ea1369f 2020-09-12T14:38:07.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:38:07.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:38:07.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:38:07.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:512: TEST_backfill_multi_partial: ceph osd in osd.1 2020-09-12T14:38:07.830 INFO:tasks.workunit.client.0.smithi027.stderr:osd.1 is already in. 2020-09-12T14:38:07.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:513: TEST_backfill_multi_partial: sleep 30 2020-09-12T14:38:37.841 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:515: TEST_backfill_multi_partial: wait_for_not_backfilling 240 2020-09-12T14:38:37.841 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:82: wait_for_not_backfilling: local timeout=240 2020-09-12T14:38:37.841 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:83: wait_for_not_backfilling: wait_for_not_state backfilling 240 2020-09-12T14:38:37.841 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:53: wait_for_not_state: local state=backfilling 2020-09-12T14:38:37.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:54: wait_for_not_state: local num_in_state=-1 2020-09-12T14:38:37.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:55: wait_for_not_state: local cur_in_state 2020-09-12T14:38:37.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:56: wait_for_not_state: delays=($(get_timeout_delays $2 5)) 2020-09-12T14:38:37.842 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:56: wait_for_not_state: get_timeout_delays 240 5 2020-09-12T14:38:37.843 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:38:37.843 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:38:37.843 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:38:37.843 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:38:37.843 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:38:37.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:56: wait_for_not_state: local -a delays 2020-09-12T14:38:37.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:57: wait_for_not_state: local -i loop=0 2020-09-12T14:38:37.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:59: wait_for_not_state: flush_pg_stats 2020-09-12T14:38:37.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:38:37.987 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:38:38.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:38:38.394 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:38:38.394 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T14:38:38.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:38:38.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:38:38.394 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:38:38.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836568 2020-09-12T14:38:38.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836568 2020-09-12T14:38:38.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836568' 2020-09-12T14:38:38.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:38:38.579 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:38:38.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=206158430217 2020-09-12T14:38:38.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 206158430217 2020-09-12T14:38:38.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836568 1-206158430217' 2020-09-12T14:38:38.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:38:38.747 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:38:38.849 INFO:tasks.workunit.client.0.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-09-12T14:38:38.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq= 2020-09-12T14:38:38.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z '' 2020-09-12T14:38:38.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2075: flush_pg_stats: continue 2020-09-12T14:38:38.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:38:38.851 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836568 2020-09-12T14:38:38.851 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:38:38.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:38:38.852 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836568 2020-09-12T14:38:38.852 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:38:38.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836568 2020-09-12T14:38:38.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836568' 2020-09-12T14:38:38.853 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836568 2020-09-12T14:38:38.854 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:38:39.261 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836566 -lt 21474836568 2020-09-12T14:38:39.261 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:38:40.262 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:38:40.262 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:38:40.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836568 -lt 21474836568 2020-09-12T14:38:40.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:38:40.696 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-206158430217 2020-09-12T14:38:40.697 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:38:40.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:38:40.697 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-206158430217 2020-09-12T14:38:40.698 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:38:40.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=206158430217 2020-09-12T14:38:40.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 206158430217' 2020-09-12T14:38:40.699 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 206158430217 2020-09-12T14:38:40.699 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:38:41.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 206158430217 -lt 206158430217 2020-09-12T14:38:41.102 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:60: wait_for_not_state: get_num_pgs 2020-09-12T14:38:41.102 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:38:41.102 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:38:41.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:60: wait_for_not_state: test 3 == 0 2020-09-12T14:38:41.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:64: wait_for_not_state: true 2020-09-12T14:38:41.634 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:65: wait_for_not_state: get_num_in_state backfilling 2020-09-12T14:38:41.634 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:44: get_num_in_state: local state=backfilling 2020-09-12T14:38:41.634 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:45: get_num_in_state: local expression 2020-09-12T14:38:41.634 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:46: get_num_in_state: expression+='select(contains("backfilling"))' 2020-09-12T14:38:41.634 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:47: get_num_in_state: ceph --format json pg dump pgs 2020-09-12T14:38:41.635 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:48: get_num_in_state: jq '.pg_stats | [.[] | .state | select(contains("backfilling"))] | length' 2020-09-12T14:38:42.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:65: wait_for_not_state: cur_in_state=1 2020-09-12T14:38:42.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:66: wait_for_not_state: test 1 = 0 2020-09-12T14:38:42.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:67: wait_for_not_state: test 1 '!=' -1 2020-09-12T14:38:42.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:68: wait_for_not_state: loop=0 2020-09-12T14:38:42.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:69: wait_for_not_state: num_in_state=1 2020-09-12T14:38:42.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:74: wait_for_not_state: sleep 5 2020-09-12T14:38:47.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:75: wait_for_not_state: loop+=1 2020-09-12T14:38:47.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:64: wait_for_not_state: true 2020-09-12T14:38:47.032 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:65: wait_for_not_state: get_num_in_state backfilling 2020-09-12T14:38:47.032 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:44: get_num_in_state: local state=backfilling 2020-09-12T14:38:47.032 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:45: get_num_in_state: local expression 2020-09-12T14:38:47.032 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:46: get_num_in_state: expression+='select(contains("backfilling"))' 2020-09-12T14:38:47.032 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:47: get_num_in_state: ceph --format json pg dump pgs 2020-09-12T14:38:47.033 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:48: get_num_in_state: jq '.pg_stats | [.[] | .state | select(contains("backfilling"))] | length' 2020-09-12T14:38:47.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:65: wait_for_not_state: cur_in_state=1 2020-09-12T14:38:47.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:66: wait_for_not_state: test 1 = 0 2020-09-12T14:38:47.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:67: wait_for_not_state: test 1 '!=' 1 2020-09-12T14:38:47.437 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:70: wait_for_not_state: (( 1 >= 17 )) 2020-09-12T14:38:47.437 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:74: wait_for_not_state: sleep 10 2020-09-12T14:38:57.437 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:75: wait_for_not_state: loop+=1 2020-09-12T14:38:57.437 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:64: wait_for_not_state: true 2020-09-12T14:38:57.438 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:65: wait_for_not_state: get_num_in_state backfilling 2020-09-12T14:38:57.438 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:44: get_num_in_state: local state=backfilling 2020-09-12T14:38:57.438 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:45: get_num_in_state: local expression 2020-09-12T14:38:57.439 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:46: get_num_in_state: expression+='select(contains("backfilling"))' 2020-09-12T14:38:57.439 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:47: get_num_in_state: ceph --format json pg dump pgs 2020-09-12T14:38:57.439 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:48: get_num_in_state: jq '.pg_stats | [.[] | .state | select(contains("backfilling"))] | length' 2020-09-12T14:38:57.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:65: wait_for_not_state: cur_in_state=1 2020-09-12T14:38:57.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:66: wait_for_not_state: test 1 = 0 2020-09-12T14:38:57.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:67: wait_for_not_state: test 1 '!=' 1 2020-09-12T14:38:57.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:70: wait_for_not_state: (( 2 >= 17 )) 2020-09-12T14:38:57.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:74: wait_for_not_state: sleep 15 2020-09-12T14:39:12.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:75: wait_for_not_state: loop+=1 2020-09-12T14:39:12.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:64: wait_for_not_state: true 2020-09-12T14:39:12.847 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:65: wait_for_not_state: get_num_in_state backfilling 2020-09-12T14:39:12.847 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:44: get_num_in_state: local state=backfilling 2020-09-12T14:39:12.847 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:45: get_num_in_state: local expression 2020-09-12T14:39:12.848 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:46: get_num_in_state: expression+='select(contains("backfilling"))' 2020-09-12T14:39:12.848 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:47: get_num_in_state: ceph --format json pg dump pgs 2020-09-12T14:39:12.848 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:48: get_num_in_state: jq '.pg_stats | [.[] | .state | select(contains("backfilling"))] | length' 2020-09-12T14:39:13.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:65: wait_for_not_state: cur_in_state=1 2020-09-12T14:39:13.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:66: wait_for_not_state: test 1 = 0 2020-09-12T14:39:13.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:67: wait_for_not_state: test 1 '!=' 1 2020-09-12T14:39:13.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:70: wait_for_not_state: (( 3 >= 17 )) 2020-09-12T14:39:13.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:74: wait_for_not_state: sleep 15 2020-09-12T14:39:28.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:75: wait_for_not_state: loop+=1 2020-09-12T14:39:28.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:64: wait_for_not_state: true 2020-09-12T14:39:28.260 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:65: wait_for_not_state: get_num_in_state backfilling 2020-09-12T14:39:28.261 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:44: get_num_in_state: local state=backfilling 2020-09-12T14:39:28.261 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:45: get_num_in_state: local expression 2020-09-12T14:39:28.261 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:46: get_num_in_state: expression+='select(contains("backfilling"))' 2020-09-12T14:39:28.261 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:47: get_num_in_state: ceph --format json pg dump pgs 2020-09-12T14:39:28.261 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:48: get_num_in_state: jq '.pg_stats | [.[] | .state | select(contains("backfilling"))] | length' 2020-09-12T14:39:28.665 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:65: wait_for_not_state: cur_in_state=1 2020-09-12T14:39:28.665 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:66: wait_for_not_state: test 1 = 0 2020-09-12T14:39:28.665 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:67: wait_for_not_state: test 1 '!=' 1 2020-09-12T14:39:28.665 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:70: wait_for_not_state: (( 4 >= 17 )) 2020-09-12T14:39:28.666 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:74: wait_for_not_state: sleep 15 2020-09-12T14:39:43.666 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:75: wait_for_not_state: loop+=1 2020-09-12T14:39:43.667 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:64: wait_for_not_state: true 2020-09-12T14:39:43.667 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:65: wait_for_not_state: get_num_in_state backfilling 2020-09-12T14:39:43.668 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:44: get_num_in_state: local state=backfilling 2020-09-12T14:39:43.668 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:45: get_num_in_state: local expression 2020-09-12T14:39:43.668 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:46: get_num_in_state: expression+='select(contains("backfilling"))' 2020-09-12T14:39:43.668 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:47: get_num_in_state: ceph --format json pg dump pgs 2020-09-12T14:39:43.668 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:48: get_num_in_state: jq '.pg_stats | [.[] | .state | select(contains("backfilling"))] | length' 2020-09-12T14:39:44.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:65: wait_for_not_state: cur_in_state=0 2020-09-12T14:39:44.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:66: wait_for_not_state: test 0 = 0 2020-09-12T14:39:44.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:66: wait_for_not_state: break 2020-09-12T14:39:44.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:77: wait_for_not_state: return 0 2020-09-12T14:39:44.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:516: TEST_backfill_multi_partial: wait_for_not_activating 60 2020-09-12T14:39:44.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:88: wait_for_not_activating: local timeout=60 2020-09-12T14:39:44.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:89: wait_for_not_activating: wait_for_not_state activating 60 2020-09-12T14:39:44.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:53: wait_for_not_state: local state=activating 2020-09-12T14:39:44.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:54: wait_for_not_state: local num_in_state=-1 2020-09-12T14:39:44.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:55: wait_for_not_state: local cur_in_state 2020-09-12T14:39:44.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:56: wait_for_not_state: delays=($(get_timeout_delays $2 5)) 2020-09-12T14:39:44.074 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:56: wait_for_not_state: get_timeout_delays 60 5 2020-09-12T14:39:44.074 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:39:44.075 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:39:44.075 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:39:44.075 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:39:44.076 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:39:44.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:56: wait_for_not_state: local -a delays 2020-09-12T14:39:44.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:57: wait_for_not_state: local -i loop=0 2020-09-12T14:39:44.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:59: wait_for_not_state: flush_pg_stats 2020-09-12T14:39:44.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:39:44.115 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:39:44.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:39:44.540 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:39:44.540 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T14:39:44.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:39:44.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:39:44.540 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:39:44.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836583 2020-09-12T14:39:44.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836583 2020-09-12T14:39:44.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836583' 2020-09-12T14:39:44.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:39:44.690 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:39:44.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=206158430232 2020-09-12T14:39:44.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 206158430232 2020-09-12T14:39:44.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836583 1-206158430232' 2020-09-12T14:39:44.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:39:44.838 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:39:44.936 INFO:tasks.workunit.client.0.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-09-12T14:39:44.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq= 2020-09-12T14:39:44.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z '' 2020-09-12T14:39:44.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2075: flush_pg_stats: continue 2020-09-12T14:39:44.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:39:44.939 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836583 2020-09-12T14:39:44.939 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:39:44.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:39:44.940 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836583 2020-09-12T14:39:44.940 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:39:44.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836583 2020-09-12T14:39:44.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836583' 2020-09-12T14:39:44.941 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836583 2020-09-12T14:39:44.941 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:39:45.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836581 -lt 21474836583 2020-09-12T14:39:45.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:39:46.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:39:46.346 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:39:46.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836583 -lt 21474836583 2020-09-12T14:39:46.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:39:46.792 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-206158430232 2020-09-12T14:39:46.793 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:39:46.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:39:46.793 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-206158430232 2020-09-12T14:39:46.794 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:39:46.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=206158430232 2020-09-12T14:39:46.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 206158430232' 2020-09-12T14:39:46.795 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 206158430232 2020-09-12T14:39:46.795 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:39:47.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 206158430232 -lt 206158430232 2020-09-12T14:39:47.202 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:60: wait_for_not_state: get_num_pgs 2020-09-12T14:39:47.202 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:39:47.202 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:39:47.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:60: wait_for_not_state: test 3 == 0 2020-09-12T14:39:47.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:64: wait_for_not_state: true 2020-09-12T14:39:47.739 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:65: wait_for_not_state: get_num_in_state activating 2020-09-12T14:39:47.739 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:44: get_num_in_state: local state=activating 2020-09-12T14:39:47.740 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:45: get_num_in_state: local expression 2020-09-12T14:39:47.740 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:46: get_num_in_state: expression+='select(contains("activating"))' 2020-09-12T14:39:47.740 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:47: get_num_in_state: ceph --format json pg dump pgs 2020-09-12T14:39:47.740 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:48: get_num_in_state: jq '.pg_stats | [.[] | .state | select(contains("activating"))] | length' 2020-09-12T14:39:48.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:65: wait_for_not_state: cur_in_state=0 2020-09-12T14:39:48.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:66: wait_for_not_state: test 0 = 0 2020-09-12T14:39:48.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:66: wait_for_not_state: break 2020-09-12T14:39:48.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:77: wait_for_not_state: return 0 2020-09-12T14:39:48.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:518: TEST_backfill_multi_partial: flush_pg_stats 2020-09-12T14:39:48.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:39:48.136 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:39:48.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:39:48.553 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:39:48.553 INFO:tasks.workunit.client.0.smithi027.stderr:2' 2020-09-12T14:39:48.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:39:48.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:39:48.554 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:39:48.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836586 2020-09-12T14:39:48.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836586 2020-09-12T14:39:48.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836586' 2020-09-12T14:39:48.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:39:48.697 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:39:48.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=206158430235 2020-09-12T14:39:48.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 206158430235 2020-09-12T14:39:48.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836586 1-206158430235' 2020-09-12T14:39:48.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:39:48.840 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:39:48.943 INFO:tasks.workunit.client.0.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-09-12T14:39:48.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq= 2020-09-12T14:39:48.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z '' 2020-09-12T14:39:48.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2075: flush_pg_stats: continue 2020-09-12T14:39:48.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:39:48.946 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836586 2020-09-12T14:39:48.946 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:39:48.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:39:48.947 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836586 2020-09-12T14:39:48.947 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:39:48.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836586 2020-09-12T14:39:48.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836586' 2020-09-12T14:39:48.948 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836586 2020-09-12T14:39:48.948 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:39:49.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836584 -lt 21474836586 2020-09-12T14:39:49.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: flush_pg_stats: sleep 1 2020-09-12T14:39:50.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: flush_pg_stats: '[' 300 -eq 0 ']' 2020-09-12T14:39:50.362 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:39:50.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836586 -lt 21474836586 2020-09-12T14:39:50.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:39:50.792 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-206158430235 2020-09-12T14:39:50.793 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:39:50.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:39:50.794 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-206158430235 2020-09-12T14:39:50.794 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:39:50.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=206158430235 2020-09-12T14:39:50.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 206158430235' 2020-09-12T14:39:50.795 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:39:50.795 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 206158430235 2020-09-12T14:39:51.197 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 206158430235 -lt 206158430235 2020-09-12T14:39:51.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:519: TEST_backfill_multi_partial: ceph pg dump pgs 2020-09-12T14:39:51.593 INFO:tasks.workunit.client.0.smithi027.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN 2020-09-12T14:39:51.593 INFO:tasks.workunit.client.0.smithi027.stdout:3.0 600 0 600 0 0 1566600 0 0 100 100 active+undersized+degraded+remapped+backfill_toofull 2020-09-12T14:39:38.731193+0000 44'600 55:678 [1,0] 1 [0] 0 0'0 2020-09-12T14:32:03.946070+0000 0'0 2020-09-12T14:32:03.946070+0000 0 2020-09-12T14:39:51.594 INFO:tasks.workunit.client.0.smithi027.stdout:1.0 600 0 0 0 0 1566600 0 0 100 100 active+clean 2020-09-12T14:38:08.216929+0000 34'600 54:649 [1] 1 [1] 1 0'0 2020-09-12T14:31:58.829524+0000 0'0 2020-09-12T14:31:58.829524+0000 0 2020-09-12T14:39:51.594 INFO:tasks.workunit.client.0.smithi027.stdout:2.0 600 0 0 0 0 1566600 0 0 10 10 active+clean 2020-09-12T14:39:35.961677+0000 44'1200 55:2486 [1,0] 1 [1,0] 1 0'0 2020-09-12T14:32:01.404494+0000 0'0 2020-09-12T14:32:01.404494+0000 0 2020-09-12T14:39:51.594 INFO:tasks.workunit.client.0.smithi027.stdout: 2020-09-12T14:39:51.594 INFO:tasks.workunit.client.0.smithi027.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/docs/master/dev/placement-group/#omap-statistics for further details. 2020-09-12T14:39:51.595 INFO:tasks.workunit.client.0.smithi027.stderr:dumped pgs 2020-09-12T14:39:51.604 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:521: TEST_backfill_multi_partial: ERRORS=0 2020-09-12T14:39:51.605 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:522: TEST_backfill_multi_partial: get_num_in_state backfill_toofull 2020-09-12T14:39:51.605 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:44: get_num_in_state: local state=backfill_toofull 2020-09-12T14:39:51.605 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:45: get_num_in_state: local expression 2020-09-12T14:39:51.605 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:46: get_num_in_state: expression+='select(contains("backfill_toofull"))' 2020-09-12T14:39:51.605 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:47: get_num_in_state: ceph --format json pg dump pgs 2020-09-12T14:39:51.606 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:48: get_num_in_state: jq '.pg_stats | [.[] | .state | select(contains("backfill_toofull"))] | length' 2020-09-12T14:39:51.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:522: TEST_backfill_multi_partial: '[' 1 '!=' 1 ']' 2020-09-12T14:39:51.997 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:528: TEST_backfill_multi_partial: get_num_in_state active+clean 2020-09-12T14:39:51.997 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:44: get_num_in_state: local state=active+clean 2020-09-12T14:39:51.997 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:45: get_num_in_state: local expression 2020-09-12T14:39:51.997 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:46: get_num_in_state: expression+='select(contains("active+clean"))' 2020-09-12T14:39:51.998 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:47: get_num_in_state: ceph --format json pg dump pgs 2020-09-12T14:39:51.998 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:48: get_num_in_state: jq '.pg_stats | [.[] | .state | select(contains("active+clean"))] | length' 2020-09-12T14:39:52.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:528: TEST_backfill_multi_partial: '[' 2 '!=' 2 ']' 2020-09-12T14:39:52.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:534: TEST_backfill_multi_partial: '[' 0 '!=' 0 ']' 2020-09-12T14:39:52.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:539: TEST_backfill_multi_partial: delete_pool fillpool 2020-09-12T14:39:52.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: delete_pool: local poolname=fillpool 2020-09-12T14:39:52.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: ceph osd pool delete fillpool fillpool --yes-i-really-really-mean-it 2020-09-12T14:39:52.920 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'fillpool' does not exist 2020-09-12T14:39:52.930 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:540: TEST_backfill_multi_partial: seq 1 2 2020-09-12T14:39:52.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:540: TEST_backfill_multi_partial: for i in $(seq 1 $pools) 2020-09-12T14:39:52.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:542: TEST_backfill_multi_partial: delete_pool test1 2020-09-12T14:39:52.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: delete_pool: local poolname=test1 2020-09-12T14:39:52.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: ceph osd pool delete test1 test1 --yes-i-really-really-mean-it 2020-09-12T14:39:53.528 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test1' does not exist 2020-09-12T14:39:53.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:540: TEST_backfill_multi_partial: for i in $(seq 1 $pools) 2020-09-12T14:39:53.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:542: TEST_backfill_multi_partial: delete_pool test2 2020-09-12T14:39:53.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: delete_pool: local poolname=test2 2020-09-12T14:39:53.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: ceph osd pool delete test2 test2 --yes-i-really-really-mean-it 2020-09-12T14:39:54.134 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test2' does not exist 2020-09-12T14:39:54.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:544: TEST_backfill_multi_partial: kill_daemons td/osd-backfill-space 2020-09-12T14:39:54.144 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:39:54.144 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:39:54.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:39:54.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:39:54.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:39:54.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:39:54.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:545: TEST_backfill_multi_partial: grep -q 'num_bytes mismatch' td/osd-backfill-space/osd.0.log td/osd-backfill-space/osd.1.log td/osd-backfill-space/osd.2.log 2020-09-12T14:39:54.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:38: run: teardown td/osd-backfill-space 2020-09-12T14:39:54.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-backfill-space 2020-09-12T14:39:54.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:39:54.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-backfill-space KILL 2020-09-12T14:39:54.296 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:39:54.296 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:39:54.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:39:54.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:39:54.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:39:54.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:39:54.301 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:39:54.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:39:54.302 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:39:54.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:39:54.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:39:54.303 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:39:54.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:39:54.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:39:54.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:39:54.305 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:39:54.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:39:54.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:39:54.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-backfill-space 2020-09-12T14:39:54.364 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:39:54.364 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:39:54.365 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:39:54.365 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.145692 2020-09-12T14:39:54.366 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:39:54.366 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:39:54.366 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:35: run: for func in $funcs 2020-09-12T14:39:54.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:36: run: setup td/osd-backfill-space 2020-09-12T14:39:54.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-backfill-space 2020-09-12T14:39:54.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-backfill-space 2020-09-12T14:39:54.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-backfill-space 2020-09-12T14:39:54.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2020-09-12T14:39:54.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-backfill-space KILL 2020-09-12T14:39:54.368 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2020-09-12T14:39:54.368 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2020-09-12T14:39:54.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2020-09-12T14:39:54.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2020-09-12T14:39:54.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2020-09-12T14:39:54.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2020-09-12T14:39:54.369 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2020-09-12T14:39:54.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2020-09-12T14:39:54.369 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2020-09-12T14:39:54.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2020-09-12T14:39:54.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2020-09-12T14:39:54.370 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2020-09-12T14:39:54.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:39:54.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2020-09-12T14:39:54.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2020-09-12T14:39:54.373 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-12T14:39:54.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2020-09-12T14:39:54.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2020-09-12T14:39:54.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-backfill-space 2020-09-12T14:39:54.375 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2020-09-12T14:39:54.375 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:39:54.375 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:39:54.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.145692 2020-09-12T14:39:54.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2020-09-12T14:39:54.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2020-09-12T14:39:54.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-backfill-space 2020-09-12T14:39:54.377 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2020-09-12T14:39:54.377 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:39:54.377 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:39:54.378 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.145692 2020-09-12T14:39:54.379 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2020-09-12T14:39:54.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2020-09-12T14:39:54.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2020-09-12T14:39:54.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-backfill-space 1' TERM HUP INT 2020-09-12T14:39:54.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:37: run: TEST_backfill_test_multi td/osd-backfill-space 2020-09-12T14:39:54.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:191: TEST_backfill_test_multi: local dir=td/osd-backfill-space 2020-09-12T14:39:54.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:192: TEST_backfill_test_multi: local pools=8 2020-09-12T14:39:54.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:193: TEST_backfill_test_multi: local OSDS=20 2020-09-12T14:39:54.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:195: TEST_backfill_test_multi: run_mon td/osd-backfill-space a 2020-09-12T14:39:54.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-backfill-space 2020-09-12T14:39:54.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2020-09-12T14:39:54.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2020-09-12T14:39:54.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2020-09-12T14:39:54.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-backfill-space/a 2020-09-12T14:39:54.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-backfill-space/a --run-dir=td/osd-backfill-space 2020-09-12T14:39:54.698 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2020-09-12T14:39:54.699 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:39:54.699 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:39:54.699 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:39:54.699 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:39:54.700 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:39:54.700 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:39:54.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-backfill-space/a '--log-file=td/osd-backfill-space/$name.log' '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --mon-cluster-log-file=td/osd-backfill-space/log --run-dir=td/osd-backfill-space '--pid-file=td/osd-backfill-space/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 2020-09-12T14:39:54.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: cat 2020-09-12T14:39:54.867 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a fsid 2020-09-12T14:39:54.867 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:39:54.868 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:39:54.868 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=fsid 2020-09-12T14:39:54.868 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .fsid 2020-09-12T14:39:54.868 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:39:54.868 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:39:54.868 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:39:54.869 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:39:54.869 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:39:54.869 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:39:54.869 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.145692/ceph-mon.a.asok 2020-09-12T14:39:54.870 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:39:54.870 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.145692/ceph-mon.a.asok config get fsid 2020-09-12T14:39:54.971 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: run_mon: get_config mon a mon_host 2020-09-12T14:39:54.972 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: get_config: local daemon=mon 2020-09-12T14:39:54.972 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: get_config: local id=a 2020-09-12T14:39:54.972 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: get_config: local config=mon_host 2020-09-12T14:39:54.972 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1070: get_config: jq -r .mon_host 2020-09-12T14:39:54.973 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: get_asok_path mon.a 2020-09-12T14:39:54.973 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2020-09-12T14:39:54.973 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2020-09-12T14:39:54.974 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2020-09-12T14:39:54.974 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:39:54.974 INFO:tasks.workunit.client.0.smithi027.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:39:54.974 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.145692/ceph-mon.a.asok 2020-09-12T14:39:54.974 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: CEPH_ARGS= 2020-09-12T14:39:54.974 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1067: get_config: ceph --format json daemon /tmp/ceph-asok.145692/ceph-mon.a.asok config get mon_host 2020-09-12T14:39:55.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:196: TEST_backfill_test_multi: run_mgr td/osd-backfill-space x 2020-09-12T14:39:55.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: local dir=td/osd-backfill-space 2020-09-12T14:39:55.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: shift 2020-09-12T14:39:55.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: local id=x 2020-09-12T14:39:55.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: shift 2020-09-12T14:39:55.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: local data=td/osd-backfill-space/x 2020-09-12T14:39:55.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:558: run_mgr: ceph config set mgr mgr/devicehealth/enable_monitoring off --force 2020-09-12T14:39:55.593 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: get_asok_path 2020-09-12T14:39:55.593 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:39:55.593 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:39:55.593 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:39:55.593 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:39:55.594 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:39:55.594 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:39:55.594 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:39:55.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:560: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-backfill-space/x '--log-file=td/osd-backfill-space/$name.log' '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --run-dir=td/osd-backfill-space '--pid-file=td/osd-backfill-space/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2020-09-12T14:39:55.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:197: TEST_backfill_test_multi: export CEPH_ARGS 2020-09-12T14:39:55.619 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:199: TEST_backfill_test_multi: expr 20 - 1 2020-09-12T14:39:55.620 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:199: TEST_backfill_test_multi: seq 0 19 2020-09-12T14:39:55.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:199: TEST_backfill_test_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:39:55.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:201: TEST_backfill_test_multi: run_osd td/osd-backfill-space 0 2020-09-12T14:39:55.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:39:55.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:39:55.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=0 2020-09-12T14:39:55.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:39:55.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/0 2020-09-12T14:39:55.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:39:55.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:39:55.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:39:55.625 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:39:55.625 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/0' 2020-09-12T14:39:55.625 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/0/journal' 2020-09-12T14:39:55.625 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:39:55.625 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:39:55.625 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:39:55.625 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:39:55.625 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:39:55.626 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:39:55.626 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:39:55.626 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:39:55.626 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:39:55.626 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:39:55.626 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:39:55.626 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:39:55.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:39:55.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:39:55.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:39:55.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:39:55.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:39:55.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:39:55.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:39:55.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:39:55.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:39:55.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/0 2020-09-12T14:39:55.628 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:39:55.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=fba09219-b82a-4e62-a9f3-9f9435dc867f 2020-09-12T14:39:55.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd0 fba09219-b82a-4e62-a9f3-9f9435dc867f' 2020-09-12T14:39:55.629 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:39:55.629 INFO:tasks.workunit.client.0.smithi027.stdout:add osd0 fba09219-b82a-4e62-a9f3-9f9435dc867f 2020-09-12T14:39:55.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQC73VxfSTZDJhAAUsnPuE9iVasGgxZuqv9+NQ== 2020-09-12T14:39:55.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQC73VxfSTZDJhAAUsnPuE9iVasGgxZuqv9+NQ=="}' 2020-09-12T14:39:55.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new fba09219-b82a-4e62-a9f3-9f9435dc867f -i td/osd-backfill-space/0/new.json 2020-09-12T14:39:55.994 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:39:56.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/0/new.json 2020-09-12T14:39:56.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 0 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/0 --osd-journal=td/osd-backfill-space/0/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQC73VxfSTZDJhAAUsnPuE9iVasGgxZuqv9+NQ== --osd-uuid fba09219-b82a-4e62-a9f3-9f9435dc867f 2020-09-12T14:39:56.034 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:39:56.033+0000 7fb97d365ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:39:56.036 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:39:56.035+0000 7fb97d365ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:39:56.037 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:39:56.036+0000 7fb97d365ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:39:56.038 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:39:56.037+0000 7fb97d365ec0 -1 bluestore(td/osd-backfill-space/0/block) _read_bdev_label failed to open td/osd-backfill-space/0/block: (2) No such file or directory 2020-09-12T14:39:56.038 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:39:56.037+0000 7fb97d365ec0 -1 bluestore(td/osd-backfill-space/0/block) _read_bdev_label failed to open td/osd-backfill-space/0/block: (2) No such file or directory 2020-09-12T14:39:56.038 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:39:56.037+0000 7fb97d365ec0 -1 bluestore(td/osd-backfill-space/0/block) _read_bdev_label failed to open td/osd-backfill-space/0/block: (2) No such file or directory 2020-09-12T14:39:56.092 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:39:56.091+0000 7fb97d365ec0 -1 bluestore(td/osd-backfill-space/0) _read_fsid unparsable uuid 2020-09-12T14:40:00.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/0/keyring 2020-09-12T14:40:00.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:40:00.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd0 key to auth repository 2020-09-12T14:40:00.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:40:00.293 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd0 key to auth repository 2020-09-12T14:40:00.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.0 2020-09-12T14:40:00.847 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.0 2020-09-12T14:40:00.848 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:40:00.848 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:40:00.848 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:40:00.848 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 0 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/0 --osd-journal=td/osd-backfill-space/0/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:40:00.873 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:00.872+0000 7f33caadfec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:00.900 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:00.899+0000 7f33caadfec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:00.901 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:00.900+0000 7f33caadfec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:01.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 0 2020-09-12T14:40:01.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:40:01.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=0 2020-09-12T14:40:01.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:40:01.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:40:01.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:40:01.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:40:01.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:40:01.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:40:01.255 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:40:01.416 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:01.415+0000 7f33caadfec0 -1 Falling back to public interface 2020-09-12T14:40:01.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:40:02.194 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:02.192+0000 7f33caadfec0 -1 osd.0 0 log_to_monitors {default=true} 2020-09-12T14:40:02.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:40:02.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:40:02.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:40:02.715 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:40:02.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.0 up' 2020-09-12T14:40:02.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:40:03.136 INFO:tasks.workunit.client.0.smithi027.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/235749,v1:127.0.0.1:6803/235749] [v2:127.0.0.1:6804/235749,v1:127.0.0.1:6805/235749] exists,up fba09219-b82a-4e62-a9f3-9f9435dc867f 2020-09-12T14:40:03.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:40:03.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:40:03.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:40:03.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:199: TEST_backfill_test_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:40:03.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:201: TEST_backfill_test_multi: run_osd td/osd-backfill-space 1 2020-09-12T14:40:03.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:40:03.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:40:03.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=1 2020-09-12T14:40:03.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:40:03.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/1 2020-09-12T14:40:03.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:40:03.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:40:03.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:40:03.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:40:03.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/1' 2020-09-12T14:40:03.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/1/journal' 2020-09-12T14:40:03.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:40:03.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:40:03.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:40:03.139 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:40:03.140 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:40:03.140 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:40:03.140 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:40:03.140 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:40:03.140 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:40:03.140 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:40:03.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:40:03.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:40:03.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:40:03.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:40:03.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:40:03.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:40:03.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:40:03.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:40:03.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:40:03.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:40:03.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:40:03.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/1 2020-09-12T14:40:03.142 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:40:03.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=f1327b9d-1f67-4637-9759-3574730d863f 2020-09-12T14:40:03.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd1 f1327b9d-1f67-4637-9759-3574730d863f' 2020-09-12T14:40:03.143 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:40:03.143 INFO:tasks.workunit.client.0.smithi027.stdout:add osd1 f1327b9d-1f67-4637-9759-3574730d863f 2020-09-12T14:40:03.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDD3Vxf60krCRAAHixcm81v51vaPDrwDtjA6A== 2020-09-12T14:40:03.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDD3Vxf60krCRAAHixcm81v51vaPDrwDtjA6A=="}' 2020-09-12T14:40:03.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new f1327b9d-1f67-4637-9759-3574730d863f -i td/osd-backfill-space/1/new.json 2020-09-12T14:40:03.681 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:40:03.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/1/new.json 2020-09-12T14:40:03.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 1 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/1 --osd-journal=td/osd-backfill-space/1/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDD3Vxf60krCRAAHixcm81v51vaPDrwDtjA6A== --osd-uuid f1327b9d-1f67-4637-9759-3574730d863f 2020-09-12T14:40:03.721 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:03.721+0000 7f46ee4a7ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:03.723 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:03.723+0000 7f46ee4a7ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:03.724 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:03.723+0000 7f46ee4a7ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:03.724 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:03.724+0000 7f46ee4a7ec0 -1 bluestore(td/osd-backfill-space/1/block) _read_bdev_label failed to open td/osd-backfill-space/1/block: (2) No such file or directory 2020-09-12T14:40:03.725 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:03.724+0000 7f46ee4a7ec0 -1 bluestore(td/osd-backfill-space/1/block) _read_bdev_label failed to open td/osd-backfill-space/1/block: (2) No such file or directory 2020-09-12T14:40:03.725 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:03.724+0000 7f46ee4a7ec0 -1 bluestore(td/osd-backfill-space/1/block) _read_bdev_label failed to open td/osd-backfill-space/1/block: (2) No such file or directory 2020-09-12T14:40:03.767 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:03.766+0000 7f46ee4a7ec0 -1 bluestore(td/osd-backfill-space/1) _read_fsid unparsable uuid 2020-09-12T14:40:07.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/1/keyring 2020-09-12T14:40:07.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:40:07.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd1 key to auth repository 2020-09-12T14:40:07.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:40:07.992 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd1 key to auth repository 2020-09-12T14:40:08.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.1 2020-09-12T14:40:08.536 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.1 2020-09-12T14:40:08.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:40:08.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:40:08.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:40:08.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 1 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/1 --osd-journal=td/osd-backfill-space/1/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:40:08.562 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:08.562+0000 7f9b7e70eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:08.589 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:08.589+0000 7f9b7e70eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:08.590 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:08.590+0000 7f9b7e70eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:08.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 1 2020-09-12T14:40:08.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:40:08.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=1 2020-09-12T14:40:08.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:40:08.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:40:08.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:40:08.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:40:08.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:40:08.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:40:08.940 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:40:09.106 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:09.105+0000 7f9b7e70eec0 -1 Falling back to public interface 2020-09-12T14:40:09.362 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:40:09.915 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:09.915+0000 7f9b7e70eec0 -1 osd.1 0 log_to_monitors {default=true} 2020-09-12T14:40:10.363 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:40:10.363 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:40:10.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:40:10.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:40:10.364 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:40:10.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.1 up' 2020-09-12T14:40:10.782 INFO:tasks.workunit.client.0.smithi027.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/236748,v1:127.0.0.1:6811/236748] [v2:127.0.0.1:6812/236748,v1:127.0.0.1:6813/236748] exists,up f1327b9d-1f67-4637-9759-3574730d863f 2020-09-12T14:40:10.782 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:40:10.782 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:40:10.782 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:40:10.782 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:199: TEST_backfill_test_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:40:10.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:201: TEST_backfill_test_multi: run_osd td/osd-backfill-space 2 2020-09-12T14:40:10.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:40:10.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:40:10.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=2 2020-09-12T14:40:10.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:40:10.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/2 2020-09-12T14:40:10.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:40:10.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:40:10.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:40:10.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:40:10.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/2' 2020-09-12T14:40:10.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/2/journal' 2020-09-12T14:40:10.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:40:10.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:40:10.785 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:40:10.785 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:40:10.785 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:40:10.785 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:40:10.785 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:40:10.785 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:40:10.785 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:40:10.785 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:40:10.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:40:10.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:40:10.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:40:10.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:40:10.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:40:10.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:40:10.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:40:10.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:40:10.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:40:10.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:40:10.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:40:10.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/2 2020-09-12T14:40:10.788 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:40:10.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=b4ecf687-6e67-476c-867b-e2338e541637 2020-09-12T14:40:10.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd2 b4ecf687-6e67-476c-867b-e2338e541637' 2020-09-12T14:40:10.789 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:40:10.789 INFO:tasks.workunit.client.0.smithi027.stdout:add osd2 b4ecf687-6e67-476c-867b-e2338e541637 2020-09-12T14:40:10.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDK3VxfhH7DLxAATJSzfXSHPgcUMQUnmC1vYg== 2020-09-12T14:40:10.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDK3VxfhH7DLxAATJSzfXSHPgcUMQUnmC1vYg=="}' 2020-09-12T14:40:10.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new b4ecf687-6e67-476c-867b-e2338e541637 -i td/osd-backfill-space/2/new.json 2020-09-12T14:40:11.349 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:40:11.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/2/new.json 2020-09-12T14:40:11.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 2 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/2 --osd-journal=td/osd-backfill-space/2/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDK3VxfhH7DLxAATJSzfXSHPgcUMQUnmC1vYg== --osd-uuid b4ecf687-6e67-476c-867b-e2338e541637 2020-09-12T14:40:11.388 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:11.388+0000 7f116fa5aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:11.415 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:11.415+0000 7f116fa5aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:11.417 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:11.417+0000 7f116fa5aec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:11.418 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:11.418+0000 7f116fa5aec0 -1 bluestore(td/osd-backfill-space/2/block) _read_bdev_label failed to open td/osd-backfill-space/2/block: (2) No such file or directory 2020-09-12T14:40:11.418 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:11.418+0000 7f116fa5aec0 -1 bluestore(td/osd-backfill-space/2/block) _read_bdev_label failed to open td/osd-backfill-space/2/block: (2) No such file or directory 2020-09-12T14:40:11.418 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:11.418+0000 7f116fa5aec0 -1 bluestore(td/osd-backfill-space/2/block) _read_bdev_label failed to open td/osd-backfill-space/2/block: (2) No such file or directory 2020-09-12T14:40:11.515 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:11.515+0000 7f116fa5aec0 -1 bluestore(td/osd-backfill-space/2) _read_fsid unparsable uuid 2020-09-12T14:40:15.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/2/keyring 2020-09-12T14:40:15.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:40:15.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd2 key to auth repository 2020-09-12T14:40:15.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:40:15.601 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd2 key to auth repository 2020-09-12T14:40:16.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.2 2020-09-12T14:40:16.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 2 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/2 --osd-journal=td/osd-backfill-space/2/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:40:16.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:40:16.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:40:16.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:40:16.155 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.2 2020-09-12T14:40:16.179 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:16.179+0000 7f3a8faf8ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:16.188 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:16.188+0000 7f3a8faf8ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:16.190 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:16.190+0000 7f3a8faf8ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:16.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 2 2020-09-12T14:40:16.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:40:16.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=2 2020-09-12T14:40:16.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:40:16.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:40:16.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:40:16.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:40:16.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:40:16.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:40:16.562 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:40:16.721 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:16.720+0000 7f3a8faf8ec0 -1 Falling back to public interface 2020-09-12T14:40:17.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:40:17.581 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:17.581+0000 7f3a8faf8ec0 -1 osd.2 0 log_to_monitors {default=true} 2020-09-12T14:40:18.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:40:18.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:40:18.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:40:18.005 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:40:18.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:40:18.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.2 up' 2020-09-12T14:40:18.449 INFO:tasks.workunit.client.0.smithi027.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/237742,v1:127.0.0.1:6819/237742] [v2:127.0.0.1:6820/237742,v1:127.0.0.1:6821/237742] exists,up b4ecf687-6e67-476c-867b-e2338e541637 2020-09-12T14:40:18.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:40:18.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:40:18.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:40:18.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:199: TEST_backfill_test_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:40:18.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:201: TEST_backfill_test_multi: run_osd td/osd-backfill-space 3 2020-09-12T14:40:18.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:40:18.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:40:18.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=3 2020-09-12T14:40:18.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:40:18.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/3 2020-09-12T14:40:18.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:40:18.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:40:18.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:40:18.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:40:18.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/3' 2020-09-12T14:40:18.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/3/journal' 2020-09-12T14:40:18.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:40:18.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:40:18.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:40:18.453 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:40:18.453 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:40:18.453 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:40:18.453 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:40:18.453 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:40:18.453 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:40:18.453 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:40:18.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:40:18.454 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:40:18.454 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:40:18.454 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:40:18.454 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:40:18.454 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:40:18.454 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:40:18.454 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:40:18.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:40:18.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:40:18.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:40:18.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/3 2020-09-12T14:40:18.456 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:40:18.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=edf49de4-98c0-435c-bee7-e4d4a128feb2 2020-09-12T14:40:18.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd3 edf49de4-98c0-435c-bee7-e4d4a128feb2' 2020-09-12T14:40:18.456 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:40:18.456 INFO:tasks.workunit.client.0.smithi027.stdout:add osd3 edf49de4-98c0-435c-bee7-e4d4a128feb2 2020-09-12T14:40:18.468 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDS3Vxf3j3rGxAAr3CY9pIK3rMgvyK1fpBb6Q== 2020-09-12T14:40:18.468 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDS3Vxf3j3rGxAAr3CY9pIK3rMgvyK1fpBb6Q=="}' 2020-09-12T14:40:18.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new edf49de4-98c0-435c-bee7-e4d4a128feb2 -i td/osd-backfill-space/3/new.json 2020-09-12T14:40:19.055 INFO:tasks.workunit.client.0.smithi027.stdout:3 2020-09-12T14:40:19.066 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/3/new.json 2020-09-12T14:40:19.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 3 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/3 --osd-journal=td/osd-backfill-space/3/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDS3Vxf3j3rGxAAr3CY9pIK3rMgvyK1fpBb6Q== --osd-uuid edf49de4-98c0-435c-bee7-e4d4a128feb2 2020-09-12T14:40:19.094 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:19.094+0000 7f22667abec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:19.096 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:19.096+0000 7f22667abec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:19.098 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:19.098+0000 7f22667abec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:19.098 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:19.098+0000 7f22667abec0 -1 bluestore(td/osd-backfill-space/3/block) _read_bdev_label failed to open td/osd-backfill-space/3/block: (2) No such file or directory 2020-09-12T14:40:19.098 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:19.098+0000 7f22667abec0 -1 bluestore(td/osd-backfill-space/3/block) _read_bdev_label failed to open td/osd-backfill-space/3/block: (2) No such file or directory 2020-09-12T14:40:19.099 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:19.098+0000 7f22667abec0 -1 bluestore(td/osd-backfill-space/3/block) _read_bdev_label failed to open td/osd-backfill-space/3/block: (2) No such file or directory 2020-09-12T14:40:19.149 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:19.149+0000 7f22667abec0 -1 bluestore(td/osd-backfill-space/3) _read_fsid unparsable uuid 2020-09-12T14:40:23.366 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/3/keyring 2020-09-12T14:40:23.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:40:23.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd3 key to auth repository 2020-09-12T14:40:23.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:40:23.368 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd3 key to auth repository 2020-09-12T14:40:23.911 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.3 2020-09-12T14:40:23.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.3 2020-09-12T14:40:23.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:40:23.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 3 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/3 --osd-journal=td/osd-backfill-space/3/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:40:23.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:40:23.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:40:23.937 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:23.936+0000 7fe954c86ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:23.938 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:23.938+0000 7fe954c86ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:23.939 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:23.939+0000 7fe954c86ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:24.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 3 2020-09-12T14:40:24.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:40:24.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=3 2020-09-12T14:40:24.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:40:24.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:40:24.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:40:24.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:40:24.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:40:24.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.3 up' 2020-09-12T14:40:24.330 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:40:24.462 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:24.461+0000 7fe954c86ec0 -1 Falling back to public interface 2020-09-12T14:40:24.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:40:25.376 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:25.375+0000 7fe954c86ec0 -1 osd.3 0 log_to_monitors {default=true} 2020-09-12T14:40:25.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:40:25.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:40:25.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:40:25.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:40:25.756 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:40:25.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.3 up' 2020-09-12T14:40:26.202 INFO:tasks.workunit.client.0.smithi027.stdout:osd.3 up in weight 1 up_from 20 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/238736,v1:127.0.0.1:6827/238736] [v2:127.0.0.1:6828/238736,v1:127.0.0.1:6829/238736] exists,up edf49de4-98c0-435c-bee7-e4d4a128feb2 2020-09-12T14:40:26.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:40:26.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:40:26.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:40:26.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:199: TEST_backfill_test_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:40:26.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:201: TEST_backfill_test_multi: run_osd td/osd-backfill-space 4 2020-09-12T14:40:26.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:40:26.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:40:26.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=4 2020-09-12T14:40:26.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:40:26.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/4 2020-09-12T14:40:26.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:40:26.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:40:26.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:40:26.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:40:26.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/4' 2020-09-12T14:40:26.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/4/journal' 2020-09-12T14:40:26.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:40:26.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:40:26.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:40:26.205 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:40:26.205 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:40:26.205 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:40:26.205 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:40:26.205 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:40:26.206 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:40:26.206 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:40:26.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:40:26.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:40:26.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:40:26.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:40:26.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:40:26.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:40:26.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:40:26.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:40:26.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:40:26.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:40:26.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:40:26.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/4 2020-09-12T14:40:26.208 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:40:26.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=354e363c-a2b5-4f06-a01d-fe852123ba79 2020-09-12T14:40:26.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd4 354e363c-a2b5-4f06-a01d-fe852123ba79' 2020-09-12T14:40:26.208 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:40:26.209 INFO:tasks.workunit.client.0.smithi027.stdout:add osd4 354e363c-a2b5-4f06-a01d-fe852123ba79 2020-09-12T14:40:26.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDa3VxfqyMpDRAASrFdT66CHnK2WG0U9xaWjg== 2020-09-12T14:40:26.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDa3VxfqyMpDRAASrFdT66CHnK2WG0U9xaWjg=="}' 2020-09-12T14:40:26.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 354e363c-a2b5-4f06-a01d-fe852123ba79 -i td/osd-backfill-space/4/new.json 2020-09-12T14:40:26.832 INFO:tasks.workunit.client.0.smithi027.stdout:4 2020-09-12T14:40:26.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/4/new.json 2020-09-12T14:40:26.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 4 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/4 --osd-journal=td/osd-backfill-space/4/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDa3VxfqyMpDRAASrFdT66CHnK2WG0U9xaWjg== --osd-uuid 354e363c-a2b5-4f06-a01d-fe852123ba79 2020-09-12T14:40:26.874 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:26.872+0000 7fa1f04c6ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:26.875 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:26.874+0000 7fa1f04c6ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:26.877 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:26.875+0000 7fa1f04c6ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:26.878 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:26.877+0000 7fa1f04c6ec0 -1 bluestore(td/osd-backfill-space/4/block) _read_bdev_label failed to open td/osd-backfill-space/4/block: (2) No such file or directory 2020-09-12T14:40:26.878 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:26.877+0000 7fa1f04c6ec0 -1 bluestore(td/osd-backfill-space/4/block) _read_bdev_label failed to open td/osd-backfill-space/4/block: (2) No such file or directory 2020-09-12T14:40:26.878 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:26.877+0000 7fa1f04c6ec0 -1 bluestore(td/osd-backfill-space/4/block) _read_bdev_label failed to open td/osd-backfill-space/4/block: (2) No such file or directory 2020-09-12T14:40:26.977 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:26.976+0000 7fa1f04c6ec0 -1 bluestore(td/osd-backfill-space/4) _read_fsid unparsable uuid 2020-09-12T14:40:31.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/4/keyring 2020-09-12T14:40:31.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:40:31.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd4 key to auth repository 2020-09-12T14:40:31.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:40:31.329 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd4 key to auth repository 2020-09-12T14:40:31.902 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.4 2020-09-12T14:40:31.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.4 2020-09-12T14:40:31.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:40:31.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:40:31.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:40:31.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 4 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/4 --osd-journal=td/osd-backfill-space/4/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:40:31.928 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:31.927+0000 7f03f0be8ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:31.929 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:31.928+0000 7f03f0be8ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:31.931 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:31.930+0000 7f03f0be8ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:32.347 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 4 2020-09-12T14:40:32.347 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:40:32.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=4 2020-09-12T14:40:32.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:40:32.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:40:32.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:40:32.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:40:32.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:40:32.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.4 up' 2020-09-12T14:40:32.349 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:40:32.450 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:32.448+0000 7f03f0be8ec0 -1 Falling back to public interface 2020-09-12T14:40:32.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:40:33.229 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:33.228+0000 7f03f0be8ec0 -1 osd.4 0 log_to_monitors {default=true} 2020-09-12T14:40:33.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:40:33.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:40:33.761 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:40:33.761 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:40:33.761 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:40:33.761 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.4 up' 2020-09-12T14:40:34.182 INFO:tasks.workunit.client.0.smithi027.stdout:osd.4 up in weight 1 up_from 25 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/239730,v1:127.0.0.1:6835/239730] [v2:127.0.0.1:6836/239730,v1:127.0.0.1:6837/239730] exists,up 354e363c-a2b5-4f06-a01d-fe852123ba79 2020-09-12T14:40:34.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:40:34.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:40:34.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:40:34.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:199: TEST_backfill_test_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:40:34.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:201: TEST_backfill_test_multi: run_osd td/osd-backfill-space 5 2020-09-12T14:40:34.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:40:34.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:40:34.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=5 2020-09-12T14:40:34.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:40:34.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/5 2020-09-12T14:40:34.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:40:34.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:40:34.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:40:34.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:40:34.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/5' 2020-09-12T14:40:34.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/5/journal' 2020-09-12T14:40:34.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:40:34.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:40:34.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:40:34.188 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:40:34.188 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:40:34.188 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:40:34.188 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:40:34.188 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:40:34.188 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:40:34.188 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:40:34.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:40:34.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:40:34.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:40:34.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:40:34.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:40:34.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:40:34.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:40:34.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:40:34.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:40:34.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:40:34.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:40:34.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/5 2020-09-12T14:40:34.190 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:40:34.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=55f8226d-b599-43af-8a4b-3c4d69d52390 2020-09-12T14:40:34.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd5 55f8226d-b599-43af-8a4b-3c4d69d52390' 2020-09-12T14:40:34.191 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:40:34.191 INFO:tasks.workunit.client.0.smithi027.stdout:add osd5 55f8226d-b599-43af-8a4b-3c4d69d52390 2020-09-12T14:40:34.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDi3VxfC5IVDBAAzW6IxEQ3Lxp9660f53yfTA== 2020-09-12T14:40:34.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDi3VxfC5IVDBAAzW6IxEQ3Lxp9660f53yfTA=="}' 2020-09-12T14:40:34.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 55f8226d-b599-43af-8a4b-3c4d69d52390 -i td/osd-backfill-space/5/new.json 2020-09-12T14:40:34.711 INFO:tasks.workunit.client.0.smithi027.stdout:5 2020-09-12T14:40:34.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/5/new.json 2020-09-12T14:40:34.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 5 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/5 --osd-journal=td/osd-backfill-space/5/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDi3VxfC5IVDBAAzW6IxEQ3Lxp9660f53yfTA== --osd-uuid 55f8226d-b599-43af-8a4b-3c4d69d52390 2020-09-12T14:40:34.751 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:34.750+0000 7f78e05eaec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:34.764 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:34.763+0000 7f78e05eaec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:34.765 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:34.764+0000 7f78e05eaec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:34.765 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:34.764+0000 7f78e05eaec0 -1 bluestore(td/osd-backfill-space/5/block) _read_bdev_label failed to open td/osd-backfill-space/5/block: (2) No such file or directory 2020-09-12T14:40:34.765 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:34.764+0000 7f78e05eaec0 -1 bluestore(td/osd-backfill-space/5/block) _read_bdev_label failed to open td/osd-backfill-space/5/block: (2) No such file or directory 2020-09-12T14:40:34.766 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:34.764+0000 7f78e05eaec0 -1 bluestore(td/osd-backfill-space/5/block) _read_bdev_label failed to open td/osd-backfill-space/5/block: (2) No such file or directory 2020-09-12T14:40:34.831 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:34.830+0000 7f78e05eaec0 -1 bluestore(td/osd-backfill-space/5) _read_fsid unparsable uuid 2020-09-12T14:40:39.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/5/keyring 2020-09-12T14:40:39.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:40:39.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd5 key to auth repository 2020-09-12T14:40:39.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:40:39.308 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd5 key to auth repository 2020-09-12T14:40:39.859 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.5 2020-09-12T14:40:39.859 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:40:39.859 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 5 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/5 --osd-journal=td/osd-backfill-space/5/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:40:39.860 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.5 2020-09-12T14:40:39.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:40:39.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:40:39.885 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:39.883+0000 7f6be68dfec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:39.891 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:39.885+0000 7f6be68dfec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:39.891 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:39.887+0000 7f6be68dfec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:40.264 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 5 2020-09-12T14:40:40.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:40:40.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=5 2020-09-12T14:40:40.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:40:40.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:40:40.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:40:40.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:40:40.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.5 up' 2020-09-12T14:40:40.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:40:40.266 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:40:40.405 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:40.404+0000 7f6be68dfec0 -1 Falling back to public interface 2020-09-12T14:40:40.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:40:41.224 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:41.223+0000 7f6be68dfec0 -1 osd.5 0 log_to_monitors {default=true} 2020-09-12T14:40:41.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:40:41.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:40:41.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:40:41.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:40:41.687 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:40:41.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.5 up' 2020-09-12T14:40:42.123 INFO:tasks.workunit.client.0.smithi027.stdout:osd.5 up in weight 1 up_from 30 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/240725,v1:127.0.0.1:6843/240725] [v2:127.0.0.1:6844/240725,v1:127.0.0.1:6845/240725] exists,up 55f8226d-b599-43af-8a4b-3c4d69d52390 2020-09-12T14:40:42.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:40:42.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:40:42.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:40:42.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:199: TEST_backfill_test_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:40:42.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:201: TEST_backfill_test_multi: run_osd td/osd-backfill-space 6 2020-09-12T14:40:42.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:40:42.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:40:42.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=6 2020-09-12T14:40:42.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:40:42.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/6 2020-09-12T14:40:42.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:40:42.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:40:42.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:40:42.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:40:42.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/6' 2020-09-12T14:40:42.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/6/journal' 2020-09-12T14:40:42.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:40:42.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:40:42.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:40:42.126 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:40:42.126 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:40:42.126 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:40:42.127 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:40:42.127 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:40:42.127 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:40:42.127 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:40:42.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:40:42.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:40:42.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:40:42.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:40:42.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:40:42.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:40:42.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:40:42.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:40:42.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:40:42.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:40:42.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:40:42.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/6 2020-09-12T14:40:42.129 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:40:42.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=070a0906-8fc8-4480-ba2d-04a9b8b4a997 2020-09-12T14:40:42.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd6 070a0906-8fc8-4480-ba2d-04a9b8b4a997' 2020-09-12T14:40:42.130 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:40:42.130 INFO:tasks.workunit.client.0.smithi027.stdout:add osd6 070a0906-8fc8-4480-ba2d-04a9b8b4a997 2020-09-12T14:40:42.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDq3Vxf6kuACBAAYeNDgcgir25c1t8akBMk9w== 2020-09-12T14:40:42.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDq3Vxf6kuACBAAYeNDgcgir25c1t8akBMk9w=="}' 2020-09-12T14:40:42.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 070a0906-8fc8-4480-ba2d-04a9b8b4a997 -i td/osd-backfill-space/6/new.json 2020-09-12T14:40:42.701 INFO:tasks.workunit.client.0.smithi027.stdout:6 2020-09-12T14:40:42.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/6/new.json 2020-09-12T14:40:42.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 6 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/6 --osd-journal=td/osd-backfill-space/6/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDq3Vxf6kuACBAAYeNDgcgir25c1t8akBMk9w== --osd-uuid 070a0906-8fc8-4480-ba2d-04a9b8b4a997 2020-09-12T14:40:42.741 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:42.740+0000 7f196a66bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:42.742 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:42.742+0000 7f196a66bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:42.744 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:42.744+0000 7f196a66bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:42.744 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:42.744+0000 7f196a66bec0 -1 bluestore(td/osd-backfill-space/6/block) _read_bdev_label failed to open td/osd-backfill-space/6/block: (2) No such file or directory 2020-09-12T14:40:42.745 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:42.744+0000 7f196a66bec0 -1 bluestore(td/osd-backfill-space/6/block) _read_bdev_label failed to open td/osd-backfill-space/6/block: (2) No such file or directory 2020-09-12T14:40:42.745 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:42.744+0000 7f196a66bec0 -1 bluestore(td/osd-backfill-space/6/block) _read_bdev_label failed to open td/osd-backfill-space/6/block: (2) No such file or directory 2020-09-12T14:40:42.782 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:42.782+0000 7f196a66bec0 -1 bluestore(td/osd-backfill-space/6) _read_fsid unparsable uuid 2020-09-12T14:40:46.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/6/keyring 2020-09-12T14:40:46.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:40:46.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd6 key to auth repository 2020-09-12T14:40:46.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:40:46.965 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd6 key to auth repository 2020-09-12T14:40:47.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.6 2020-09-12T14:40:47.514 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.6 2020-09-12T14:40:47.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:40:47.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 6 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/6 --osd-journal=td/osd-backfill-space/6/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:40:47.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:40:47.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:40:47.540 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:47.540+0000 7f2a1a281ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:47.541 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:47.541+0000 7f2a1a281ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:47.543 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:47.543+0000 7f2a1a281ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:47.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 6 2020-09-12T14:40:47.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:40:47.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=6 2020-09-12T14:40:47.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:40:47.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:40:47.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:40:47.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:40:47.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:40:47.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.6 up' 2020-09-12T14:40:47.924 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:40:48.059 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:48.059+0000 7f2a1a281ec0 -1 Falling back to public interface 2020-09-12T14:40:48.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:40:48.878 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:48.878+0000 7f2a1a281ec0 -1 osd.6 0 log_to_monitors {default=true} 2020-09-12T14:40:49.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:40:49.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:40:49.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:40:49.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:40:49.334 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:40:49.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.6 up' 2020-09-12T14:40:49.769 INFO:tasks.workunit.client.0.smithi027.stdout:osd.6 up in weight 1 up_from 35 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/241719,v1:127.0.0.1:6851/241719] [v2:127.0.0.1:6852/241719,v1:127.0.0.1:6853/241719] exists,up 070a0906-8fc8-4480-ba2d-04a9b8b4a997 2020-09-12T14:40:49.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:40:49.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:40:49.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:40:49.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:199: TEST_backfill_test_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:40:49.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:201: TEST_backfill_test_multi: run_osd td/osd-backfill-space 7 2020-09-12T14:40:49.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:40:49.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:40:49.771 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=7 2020-09-12T14:40:49.771 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:40:49.771 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/7 2020-09-12T14:40:49.771 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:40:49.771 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:40:49.771 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:40:49.771 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:40:49.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/7' 2020-09-12T14:40:49.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/7/journal' 2020-09-12T14:40:49.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:40:49.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:40:49.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:40:49.772 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:40:49.772 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:40:49.773 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:40:49.773 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:40:49.773 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:40:49.773 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:40:49.773 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:40:49.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:40:49.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:40:49.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:40:49.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:40:49.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:40:49.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:40:49.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:40:49.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:40:49.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:40:49.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:40:49.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:40:49.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/7 2020-09-12T14:40:49.775 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:40:49.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=681cf303-47d4-4f52-bd88-3376bebca198 2020-09-12T14:40:49.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd7 681cf303-47d4-4f52-bd88-3376bebca198' 2020-09-12T14:40:49.776 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:40:49.776 INFO:tasks.workunit.client.0.smithi027.stdout:add osd7 681cf303-47d4-4f52-bd88-3376bebca198 2020-09-12T14:40:49.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQDx3VxfKO73LhAA6Fr4S88UvFuPp+1G0zBYyw== 2020-09-12T14:40:49.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQDx3VxfKO73LhAA6Fr4S88UvFuPp+1G0zBYyw=="}' 2020-09-12T14:40:49.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 681cf303-47d4-4f52-bd88-3376bebca198 -i td/osd-backfill-space/7/new.json 2020-09-12T14:40:50.374 INFO:tasks.workunit.client.0.smithi027.stdout:7 2020-09-12T14:40:50.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/7/new.json 2020-09-12T14:40:50.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 7 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/7 --osd-journal=td/osd-backfill-space/7/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDx3VxfKO73LhAA6Fr4S88UvFuPp+1G0zBYyw== --osd-uuid 681cf303-47d4-4f52-bd88-3376bebca198 2020-09-12T14:40:50.414 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:50.413+0000 7fb92f9ecec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:50.415 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:50.415+0000 7fb92f9ecec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:50.416 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:50.416+0000 7fb92f9ecec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:50.417 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:50.417+0000 7fb92f9ecec0 -1 bluestore(td/osd-backfill-space/7/block) _read_bdev_label failed to open td/osd-backfill-space/7/block: (2) No such file or directory 2020-09-12T14:40:50.417 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:50.417+0000 7fb92f9ecec0 -1 bluestore(td/osd-backfill-space/7/block) _read_bdev_label failed to open td/osd-backfill-space/7/block: (2) No such file or directory 2020-09-12T14:40:50.417 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:50.417+0000 7fb92f9ecec0 -1 bluestore(td/osd-backfill-space/7/block) _read_bdev_label failed to open td/osd-backfill-space/7/block: (2) No such file or directory 2020-09-12T14:40:50.449 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:50.449+0000 7fb92f9ecec0 -1 bluestore(td/osd-backfill-space/7) _read_fsid unparsable uuid 2020-09-12T14:40:54.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/7/keyring 2020-09-12T14:40:54.894 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:40:54.894 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd7 key to auth repository 2020-09-12T14:40:54.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd7 key to auth repository 2020-09-12T14:40:54.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:40:55.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.7 2020-09-12T14:40:55.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 7 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/7 --osd-journal=td/osd-backfill-space/7/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:40:55.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:40:55.464 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.7 2020-09-12T14:40:55.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:40:55.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:40:55.493 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:55.489+0000 7f24a7e4eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:55.493 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:55.491+0000 7f24a7e4eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:55.493 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:55.492+0000 7f24a7e4eec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:55.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 7 2020-09-12T14:40:55.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:40:55.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=7 2020-09-12T14:40:55.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:40:55.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:40:55.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:40:55.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:40:55.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.7 up' 2020-09-12T14:40:55.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:40:55.874 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:40:56.017 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:56.016+0000 7f24a7e4eec0 -1 Falling back to public interface 2020-09-12T14:40:56.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:40:56.829 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:56.829+0000 7f24a7e4eec0 -1 osd.7 0 log_to_monitors {default=true} 2020-09-12T14:40:57.284 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:40:57.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:40:57.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:40:57.285 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:40:57.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:40:57.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.7 up' 2020-09-12T14:40:57.753 INFO:tasks.workunit.client.0.smithi027.stdout:osd.7 up in weight 1 up_from 40 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/242723,v1:127.0.0.1:6859/242723] [v2:127.0.0.1:6860/242723,v1:127.0.0.1:6861/242723] exists,up 681cf303-47d4-4f52-bd88-3376bebca198 2020-09-12T14:40:57.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:40:57.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:40:57.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:40:57.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:199: TEST_backfill_test_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:40:57.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:201: TEST_backfill_test_multi: run_osd td/osd-backfill-space 8 2020-09-12T14:40:57.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:40:57.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:40:57.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=8 2020-09-12T14:40:57.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:40:57.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/8 2020-09-12T14:40:57.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:40:57.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:40:57.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:40:57.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:40:57.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/8' 2020-09-12T14:40:57.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/8/journal' 2020-09-12T14:40:57.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:40:57.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:40:57.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:40:57.756 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:40:57.756 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:40:57.757 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:40:57.757 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:40:57.757 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:40:57.757 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:40:57.757 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:40:57.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:40:57.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:40:57.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:40:57.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:40:57.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:40:57.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:40:57.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:40:57.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:40:57.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:40:57.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:40:57.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:40:57.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/8 2020-09-12T14:40:57.759 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:40:57.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=40c5f4e7-4974-4bf7-9f4a-a6107d66700e 2020-09-12T14:40:57.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd8 40c5f4e7-4974-4bf7-9f4a-a6107d66700e' 2020-09-12T14:40:57.760 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:40:57.760 INFO:tasks.workunit.client.0.smithi027.stdout:add osd8 40c5f4e7-4974-4bf7-9f4a-a6107d66700e 2020-09-12T14:40:57.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQD53VxffukALhAAGhVEXkljSWhqlx5Wrq1yng== 2020-09-12T14:40:57.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQD53VxffukALhAAGhVEXkljSWhqlx5Wrq1yng=="}' 2020-09-12T14:40:57.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 40c5f4e7-4974-4bf7-9f4a-a6107d66700e -i td/osd-backfill-space/8/new.json 2020-09-12T14:40:58.339 INFO:tasks.workunit.client.0.smithi027.stdout:8 2020-09-12T14:40:58.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/8/new.json 2020-09-12T14:40:58.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 8 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/8 --osd-journal=td/osd-backfill-space/8/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQD53VxffukALhAAGhVEXkljSWhqlx5Wrq1yng== --osd-uuid 40c5f4e7-4974-4bf7-9f4a-a6107d66700e 2020-09-12T14:40:58.379 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:58.379+0000 7fb341f14ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:58.381 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:58.381+0000 7fb341f14ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:58.382 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:58.382+0000 7fb341f14ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:40:58.383 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:58.383+0000 7fb341f14ec0 -1 bluestore(td/osd-backfill-space/8/block) _read_bdev_label failed to open td/osd-backfill-space/8/block: (2) No such file or directory 2020-09-12T14:40:58.383 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:58.383+0000 7fb341f14ec0 -1 bluestore(td/osd-backfill-space/8/block) _read_bdev_label failed to open td/osd-backfill-space/8/block: (2) No such file or directory 2020-09-12T14:40:58.383 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:58.383+0000 7fb341f14ec0 -1 bluestore(td/osd-backfill-space/8/block) _read_bdev_label failed to open td/osd-backfill-space/8/block: (2) No such file or directory 2020-09-12T14:40:58.413 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:40:58.413+0000 7fb341f14ec0 -1 bluestore(td/osd-backfill-space/8) _read_fsid unparsable uuid 2020-09-12T14:41:02.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/8/keyring 2020-09-12T14:41:02.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:41:02.870 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd8 key to auth repository 2020-09-12T14:41:02.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd8 key to auth repository 2020-09-12T14:41:02.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:41:03.413 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.8 2020-09-12T14:41:03.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.8 2020-09-12T14:41:03.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 8 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/8 --osd-journal=td/osd-backfill-space/8/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:41:03.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:41:03.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:41:03.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:41:03.438 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:03.437+0000 7f3926138ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:03.460 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:03.459+0000 7f3926138ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:03.462 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:03.461+0000 7f3926138ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:03.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 8 2020-09-12T14:41:03.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:41:03.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=8 2020-09-12T14:41:03.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:41:03.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:41:03.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:41:03.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:41:03.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:41:03.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.8 up' 2020-09-12T14:41:03.883 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:41:03.986 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:03.984+0000 7f3926138ec0 -1 Falling back to public interface 2020-09-12T14:41:04.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:41:04.807 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:04.806+0000 7f3926138ec0 -1 osd.8 0 log_to_monitors {default=true} 2020-09-12T14:41:05.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:41:05.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:41:05.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:41:05.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:41:05.296 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:41:05.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.8 up' 2020-09-12T14:41:05.720 INFO:tasks.workunit.client.0.smithi027.stdout:osd.8 up in weight 1 up_from 45 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/243718,v1:127.0.0.1:6867/243718] [v2:127.0.0.1:6868/243718,v1:127.0.0.1:6869/243718] exists,up 40c5f4e7-4974-4bf7-9f4a-a6107d66700e 2020-09-12T14:41:05.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:41:05.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:41:05.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:41:05.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:199: TEST_backfill_test_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:41:05.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:201: TEST_backfill_test_multi: run_osd td/osd-backfill-space 9 2020-09-12T14:41:05.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:41:05.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:41:05.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=9 2020-09-12T14:41:05.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:41:05.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/9 2020-09-12T14:41:05.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:41:05.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:41:05.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:41:05.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:41:05.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/9' 2020-09-12T14:41:05.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/9/journal' 2020-09-12T14:41:05.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:41:05.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:41:05.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:41:05.723 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:41:05.723 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:41:05.723 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:41:05.724 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:41:05.724 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:41:05.724 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:41:05.724 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:41:05.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:41:05.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:41:05.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:41:05.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:41:05.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:41:05.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:41:05.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:41:05.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:41:05.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:41:05.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:41:05.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:41:05.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/9 2020-09-12T14:41:05.727 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:41:05.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=0295f0a1-e16a-42a5-a155-804a0a793f71 2020-09-12T14:41:05.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd9 0295f0a1-e16a-42a5-a155-804a0a793f71' 2020-09-12T14:41:05.727 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:41:05.728 INFO:tasks.workunit.client.0.smithi027.stdout:add osd9 0295f0a1-e16a-42a5-a155-804a0a793f71 2020-09-12T14:41:05.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQAB3lxfIEsgLBAAcHOKBZOQVxTq2iOfYulSVg== 2020-09-12T14:41:05.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQAB3lxfIEsgLBAAcHOKBZOQVxTq2iOfYulSVg=="}' 2020-09-12T14:41:05.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 0295f0a1-e16a-42a5-a155-804a0a793f71 -i td/osd-backfill-space/9/new.json 2020-09-12T14:41:06.282 INFO:tasks.workunit.client.0.smithi027.stdout:9 2020-09-12T14:41:06.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/9/new.json 2020-09-12T14:41:06.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 9 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/9 --osd-journal=td/osd-backfill-space/9/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQAB3lxfIEsgLBAAcHOKBZOQVxTq2iOfYulSVg== --osd-uuid 0295f0a1-e16a-42a5-a155-804a0a793f71 2020-09-12T14:41:06.322 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:06.320+0000 7f18e5f4fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:06.323 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:06.322+0000 7f18e5f4fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:06.324 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:06.323+0000 7f18e5f4fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:06.325 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:06.324+0000 7f18e5f4fec0 -1 bluestore(td/osd-backfill-space/9/block) _read_bdev_label failed to open td/osd-backfill-space/9/block: (2) No such file or directory 2020-09-12T14:41:06.326 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:06.324+0000 7f18e5f4fec0 -1 bluestore(td/osd-backfill-space/9/block) _read_bdev_label failed to open td/osd-backfill-space/9/block: (2) No such file or directory 2020-09-12T14:41:06.326 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:06.324+0000 7f18e5f4fec0 -1 bluestore(td/osd-backfill-space/9/block) _read_bdev_label failed to open td/osd-backfill-space/9/block: (2) No such file or directory 2020-09-12T14:41:06.367 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:06.366+0000 7f18e5f4fec0 -1 bluestore(td/osd-backfill-space/9) _read_fsid unparsable uuid 2020-09-12T14:41:10.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/9/keyring 2020-09-12T14:41:10.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:41:10.845 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd9 key to auth repository 2020-09-12T14:41:10.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd9 key to auth repository 2020-09-12T14:41:10.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/9/keyring auth add osd.9 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:41:11.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.9 2020-09-12T14:41:11.413 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.9 2020-09-12T14:41:11.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 9 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/9 --osd-journal=td/osd-backfill-space/9/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:41:11.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:41:11.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:41:11.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:41:11.439 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:11.438+0000 7fa04a14dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:11.460 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:11.459+0000 7fa04a14dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:11.462 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:11.460+0000 7fa04a14dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:11.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 9 2020-09-12T14:41:11.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:41:11.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=9 2020-09-12T14:41:11.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:41:11.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:41:11.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:41:11.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:41:11.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:41:11.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.9 up' 2020-09-12T14:41:11.840 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:41:11.985 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:11.983+0000 7fa04a14dec0 -1 Falling back to public interface 2020-09-12T14:41:12.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:41:12.787 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:12.785+0000 7fa04a14dec0 -1 osd.9 0 log_to_monitors {default=true} 2020-09-12T14:41:13.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:41:13.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:41:13.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:41:13.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:41:13.249 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:41:13.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.9 up' 2020-09-12T14:41:13.705 INFO:tasks.workunit.client.0.smithi027.stdout:osd.9 up in weight 1 up_from 50 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6874/244715,v1:127.0.0.1:6875/244715] [v2:127.0.0.1:6876/244715,v1:127.0.0.1:6877/244715] exists,up 0295f0a1-e16a-42a5-a155-804a0a793f71 2020-09-12T14:41:13.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:41:13.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:41:13.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:41:13.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:199: TEST_backfill_test_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:41:13.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:201: TEST_backfill_test_multi: run_osd td/osd-backfill-space 10 2020-09-12T14:41:13.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:41:13.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:41:13.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=10 2020-09-12T14:41:13.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:41:13.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/10 2020-09-12T14:41:13.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:41:13.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:41:13.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:41:13.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:41:13.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/10' 2020-09-12T14:41:13.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/10/journal' 2020-09-12T14:41:13.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:41:13.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:41:13.708 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:41:13.708 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:41:13.708 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:41:13.708 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:41:13.708 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:41:13.709 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:41:13.709 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:41:13.709 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:41:13.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:41:13.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:41:13.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:41:13.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:41:13.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:41:13.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:41:13.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:41:13.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:41:13.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:41:13.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:41:13.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:41:13.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/10 2020-09-12T14:41:13.711 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:41:13.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=c7e6ca6c-a478-4d2b-82e3-747be6c3dc95 2020-09-12T14:41:13.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd10 c7e6ca6c-a478-4d2b-82e3-747be6c3dc95' 2020-09-12T14:41:13.712 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:41:13.712 INFO:tasks.workunit.client.0.smithi027.stdout:add osd10 c7e6ca6c-a478-4d2b-82e3-747be6c3dc95 2020-09-12T14:41:13.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQAJ3lxfYGkoKxAAlysNNxegJze1vWOGGrnxUA== 2020-09-12T14:41:13.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQAJ3lxfYGkoKxAAlysNNxegJze1vWOGGrnxUA=="}' 2020-09-12T14:41:13.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new c7e6ca6c-a478-4d2b-82e3-747be6c3dc95 -i td/osd-backfill-space/10/new.json 2020-09-12T14:41:14.303 INFO:tasks.workunit.client.0.smithi027.stdout:10 2020-09-12T14:41:14.315 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/10/new.json 2020-09-12T14:41:14.315 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 10 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/10 --osd-journal=td/osd-backfill-space/10/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQAJ3lxfYGkoKxAAlysNNxegJze1vWOGGrnxUA== --osd-uuid c7e6ca6c-a478-4d2b-82e3-747be6c3dc95 2020-09-12T14:41:14.343 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:14.342+0000 7f2279d68ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:14.380 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:14.379+0000 7f2279d68ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:14.381 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:14.380+0000 7f2279d68ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:14.382 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:14.381+0000 7f2279d68ec0 -1 bluestore(td/osd-backfill-space/10/block) _read_bdev_label failed to open td/osd-backfill-space/10/block: (2) No such file or directory 2020-09-12T14:41:14.382 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:14.381+0000 7f2279d68ec0 -1 bluestore(td/osd-backfill-space/10/block) _read_bdev_label failed to open td/osd-backfill-space/10/block: (2) No such file or directory 2020-09-12T14:41:14.383 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:14.381+0000 7f2279d68ec0 -1 bluestore(td/osd-backfill-space/10/block) _read_bdev_label failed to open td/osd-backfill-space/10/block: (2) No such file or directory 2020-09-12T14:41:14.480 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:14.480+0000 7f2279d68ec0 -1 bluestore(td/osd-backfill-space/10) _read_fsid unparsable uuid 2020-09-12T14:41:19.236 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/10/keyring 2020-09-12T14:41:19.236 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:41:19.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd10 key to auth repository 2020-09-12T14:41:19.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/10/keyring auth add osd.10 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:41:19.238 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd10 key to auth repository 2020-09-12T14:41:19.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.10 2020-09-12T14:41:19.795 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.10 2020-09-12T14:41:19.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:41:19.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:41:19.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:41:19.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 10 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/10 --osd-journal=td/osd-backfill-space/10/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:41:19.821 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:19.820+0000 7f7c3afd6ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:19.836 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:19.835+0000 7f7c3afd6ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:19.836 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:19.836+0000 7f7c3afd6ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:20.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 10 2020-09-12T14:41:20.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:41:20.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=10 2020-09-12T14:41:20.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:41:20.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:41:20.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:41:20.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:41:20.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:41:20.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.10 up' 2020-09-12T14:41:20.213 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:41:20.359 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:20.359+0000 7f7c3afd6ec0 -1 Falling back to public interface 2020-09-12T14:41:20.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:41:21.153 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:21.153+0000 7f7c3afd6ec0 -1 osd.10 0 log_to_monitors {default=true} 2020-09-12T14:41:21.644 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:41:21.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:41:21.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:41:21.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:41:21.645 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:41:21.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.10 up' 2020-09-12T14:41:22.070 INFO:tasks.workunit.client.0.smithi027.stdout:osd.10 up in weight 1 up_from 55 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6882/245709,v1:127.0.0.1:6883/245709] [v2:127.0.0.1:6884/245709,v1:127.0.0.1:6885/245709] exists,up c7e6ca6c-a478-4d2b-82e3-747be6c3dc95 2020-09-12T14:41:22.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:41:22.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:41:22.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:41:22.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:199: TEST_backfill_test_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:41:22.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:201: TEST_backfill_test_multi: run_osd td/osd-backfill-space 11 2020-09-12T14:41:22.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:41:22.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:41:22.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=11 2020-09-12T14:41:22.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:41:22.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/11 2020-09-12T14:41:22.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:41:22.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:41:22.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:41:22.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:41:22.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/11' 2020-09-12T14:41:22.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/11/journal' 2020-09-12T14:41:22.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:41:22.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:41:22.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:41:22.073 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:41:22.074 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:41:22.074 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:41:22.074 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:41:22.074 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:41:22.074 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:41:22.074 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:41:22.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:41:22.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:41:22.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:41:22.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:41:22.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:41:22.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:41:22.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:41:22.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:41:22.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:41:22.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:41:22.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:41:22.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/11 2020-09-12T14:41:22.076 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:41:22.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=41b12981-eecf-4e7d-a442-7ccb16a98df0 2020-09-12T14:41:22.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd11 41b12981-eecf-4e7d-a442-7ccb16a98df0' 2020-09-12T14:41:22.077 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:41:22.077 INFO:tasks.workunit.client.0.smithi027.stdout:add osd11 41b12981-eecf-4e7d-a442-7ccb16a98df0 2020-09-12T14:41:22.089 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQAS3lxfkbJPBRAA+/w/tHn2WK8kLOOEkqdNRw== 2020-09-12T14:41:22.089 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQAS3lxfkbJPBRAA+/w/tHn2WK8kLOOEkqdNRw=="}' 2020-09-12T14:41:22.089 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 41b12981-eecf-4e7d-a442-7ccb16a98df0 -i td/osd-backfill-space/11/new.json 2020-09-12T14:41:22.596 INFO:tasks.workunit.client.0.smithi027.stdout:11 2020-09-12T14:41:22.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/11/new.json 2020-09-12T14:41:22.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 11 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/11 --osd-journal=td/osd-backfill-space/11/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQAS3lxfkbJPBRAA+/w/tHn2WK8kLOOEkqdNRw== --osd-uuid 41b12981-eecf-4e7d-a442-7ccb16a98df0 2020-09-12T14:41:22.636 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:22.636+0000 7fa990352ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:22.638 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:22.638+0000 7fa990352ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:22.639 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:22.639+0000 7fa990352ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:22.640 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:22.639+0000 7fa990352ec0 -1 bluestore(td/osd-backfill-space/11/block) _read_bdev_label failed to open td/osd-backfill-space/11/block: (2) No such file or directory 2020-09-12T14:41:22.640 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:22.640+0000 7fa990352ec0 -1 bluestore(td/osd-backfill-space/11/block) _read_bdev_label failed to open td/osd-backfill-space/11/block: (2) No such file or directory 2020-09-12T14:41:22.640 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:22.640+0000 7fa990352ec0 -1 bluestore(td/osd-backfill-space/11/block) _read_bdev_label failed to open td/osd-backfill-space/11/block: (2) No such file or directory 2020-09-12T14:41:22.680 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:22.680+0000 7fa990352ec0 -1 bluestore(td/osd-backfill-space/11) _read_fsid unparsable uuid 2020-09-12T14:41:27.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/11/keyring 2020-09-12T14:41:27.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:41:27.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd11 key to auth repository 2020-09-12T14:41:27.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/11/keyring auth add osd.11 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:41:27.188 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd11 key to auth repository 2020-09-12T14:41:27.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.11 2020-09-12T14:41:27.735 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.11 2020-09-12T14:41:27.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 11 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/11 --osd-journal=td/osd-backfill-space/11/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:41:27.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:41:27.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:41:27.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:41:27.760 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:27.760+0000 7fbd19e14ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:27.787 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:27.786+0000 7fbd19e14ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:27.788 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:27.788+0000 7fbd19e14ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:28.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 11 2020-09-12T14:41:28.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:41:28.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=11 2020-09-12T14:41:28.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:41:28.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:41:28.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:41:28.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:41:28.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:41:28.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.11 up' 2020-09-12T14:41:28.147 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:41:28.305 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:28.304+0000 7fbd19e14ec0 -1 Falling back to public interface 2020-09-12T14:41:28.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:41:29.114 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:29.114+0000 7fbd19e14ec0 -1 osd.11 0 log_to_monitors {default=true} 2020-09-12T14:41:29.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:41:29.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:41:29.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:41:29.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:41:29.570 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:41:29.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.11 up' 2020-09-12T14:41:30.047 INFO:tasks.workunit.client.0.smithi027.stdout:osd.11 up in weight 1 up_from 60 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6890/246707,v1:127.0.0.1:6891/246707] [v2:127.0.0.1:6892/246707,v1:127.0.0.1:6893/246707] exists,up 41b12981-eecf-4e7d-a442-7ccb16a98df0 2020-09-12T14:41:30.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:41:30.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:41:30.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:41:30.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:199: TEST_backfill_test_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:41:30.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:201: TEST_backfill_test_multi: run_osd td/osd-backfill-space 12 2020-09-12T14:41:30.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:41:30.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:41:30.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=12 2020-09-12T14:41:30.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:41:30.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/12 2020-09-12T14:41:30.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:41:30.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:41:30.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:41:30.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:41:30.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/12' 2020-09-12T14:41:30.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/12/journal' 2020-09-12T14:41:30.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:41:30.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:41:30.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:41:30.050 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:41:30.051 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:41:30.051 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:41:30.051 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:41:30.051 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:41:30.051 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:41:30.051 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:41:30.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:41:30.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:41:30.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:41:30.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:41:30.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:41:30.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:41:30.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:41:30.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:41:30.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:41:30.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:41:30.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:41:30.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/12 2020-09-12T14:41:30.053 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:41:30.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=964d9bcf-38a7-4437-b69a-ab9347851251 2020-09-12T14:41:30.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd12 964d9bcf-38a7-4437-b69a-ab9347851251' 2020-09-12T14:41:30.054 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:41:30.054 INFO:tasks.workunit.client.0.smithi027.stdout:add osd12 964d9bcf-38a7-4437-b69a-ab9347851251 2020-09-12T14:41:30.066 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQAa3lxfSMvqAxAAAwkMRPR+r16i2wvwMVQf3w== 2020-09-12T14:41:30.066 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQAa3lxfSMvqAxAAAwkMRPR+r16i2wvwMVQf3w=="}' 2020-09-12T14:41:30.066 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 964d9bcf-38a7-4437-b69a-ab9347851251 -i td/osd-backfill-space/12/new.json 2020-09-12T14:41:30.573 INFO:tasks.workunit.client.0.smithi027.stdout:12 2020-09-12T14:41:30.584 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/12/new.json 2020-09-12T14:41:30.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 12 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/12 --osd-journal=td/osd-backfill-space/12/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQAa3lxfSMvqAxAAAwkMRPR+r16i2wvwMVQf3w== --osd-uuid 964d9bcf-38a7-4437-b69a-ab9347851251 2020-09-12T14:41:30.614 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:30.614+0000 7fb55ce94ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:30.691 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:30.691+0000 7fb55ce94ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:30.692 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:30.692+0000 7fb55ce94ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:30.693 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:30.693+0000 7fb55ce94ec0 -1 bluestore(td/osd-backfill-space/12/block) _read_bdev_label failed to open td/osd-backfill-space/12/block: (2) No such file or directory 2020-09-12T14:41:30.693 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:30.693+0000 7fb55ce94ec0 -1 bluestore(td/osd-backfill-space/12/block) _read_bdev_label failed to open td/osd-backfill-space/12/block: (2) No such file or directory 2020-09-12T14:41:30.693 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:30.693+0000 7fb55ce94ec0 -1 bluestore(td/osd-backfill-space/12/block) _read_bdev_label failed to open td/osd-backfill-space/12/block: (2) No such file or directory 2020-09-12T14:41:30.817 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:30.817+0000 7fb55ce94ec0 -1 bluestore(td/osd-backfill-space/12) _read_fsid unparsable uuid 2020-09-12T14:41:35.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/12/keyring 2020-09-12T14:41:35.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:41:35.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd12 key to auth repository 2020-09-12T14:41:35.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/12/keyring auth add osd.12 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:41:35.304 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd12 key to auth repository 2020-09-12T14:41:35.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.12 2020-09-12T14:41:35.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 12 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/12 --osd-journal=td/osd-backfill-space/12/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:41:35.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:41:35.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:41:35.854 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.12 2020-09-12T14:41:35.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:41:35.879 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:35.879+0000 7f940b585ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:35.881 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:35.881+0000 7f940b585ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:35.882 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:35.882+0000 7f940b585ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:36.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 12 2020-09-12T14:41:36.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:41:36.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=12 2020-09-12T14:41:36.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:41:36.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:41:36.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:41:36.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:41:36.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:41:36.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.12 up' 2020-09-12T14:41:36.267 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:41:36.404 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:36.404+0000 7f940b585ec0 -1 Falling back to public interface 2020-09-12T14:41:36.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:41:37.222 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:37.222+0000 7f940b585ec0 -1 osd.12 0 log_to_monitors {default=true} 2020-09-12T14:41:37.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:41:37.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:41:37.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:41:37.691 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:41:37.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:41:37.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.12 up' 2020-09-12T14:41:38.201 INFO:tasks.workunit.client.0.smithi027.stdout:osd.12 up in weight 1 up_from 65 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6898/247701,v1:127.0.0.1:6899/247701] [v2:127.0.0.1:6900/247701,v1:127.0.0.1:6901/247701] exists,up 964d9bcf-38a7-4437-b69a-ab9347851251 2020-09-12T14:41:38.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:41:38.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:41:38.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:41:38.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:199: TEST_backfill_test_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:41:38.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:201: TEST_backfill_test_multi: run_osd td/osd-backfill-space 13 2020-09-12T14:41:38.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:41:38.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:41:38.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=13 2020-09-12T14:41:38.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:41:38.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/13 2020-09-12T14:41:38.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:41:38.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:41:38.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:41:38.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:41:38.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/13' 2020-09-12T14:41:38.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/13/journal' 2020-09-12T14:41:38.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:41:38.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:41:38.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:41:38.204 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:41:38.204 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:41:38.204 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:41:38.204 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:41:38.204 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:41:38.205 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:41:38.205 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:41:38.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:41:38.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:41:38.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:41:38.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:41:38.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:41:38.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:41:38.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:41:38.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:41:38.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:41:38.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:41:38.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:41:38.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/13 2020-09-12T14:41:38.208 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:41:38.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=72ab1707-92b4-419e-b071-e8165a7f8af6 2020-09-12T14:41:38.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd13 72ab1707-92b4-419e-b071-e8165a7f8af6' 2020-09-12T14:41:38.208 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:41:38.208 INFO:tasks.workunit.client.0.smithi027.stdout:add osd13 72ab1707-92b4-419e-b071-e8165a7f8af6 2020-09-12T14:41:38.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQAi3lxftPUjDRAAebm1wRwZS4VNFb5aNa95Tg== 2020-09-12T14:41:38.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQAi3lxftPUjDRAAebm1wRwZS4VNFb5aNa95Tg=="}' 2020-09-12T14:41:38.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 72ab1707-92b4-419e-b071-e8165a7f8af6 -i td/osd-backfill-space/13/new.json 2020-09-12T14:41:38.741 INFO:tasks.workunit.client.0.smithi027.stdout:13 2020-09-12T14:41:38.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/13/new.json 2020-09-12T14:41:38.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 13 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/13 --osd-journal=td/osd-backfill-space/13/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQAi3lxftPUjDRAAebm1wRwZS4VNFb5aNa95Tg== --osd-uuid 72ab1707-92b4-419e-b071-e8165a7f8af6 2020-09-12T14:41:38.782 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:38.782+0000 7f29212cfec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:38.813 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:38.813+0000 7f29212cfec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:38.813 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:38.813+0000 7f29212cfec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:38.814 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:38.814+0000 7f29212cfec0 -1 bluestore(td/osd-backfill-space/13/block) _read_bdev_label failed to open td/osd-backfill-space/13/block: (2) No such file or directory 2020-09-12T14:41:38.814 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:38.814+0000 7f29212cfec0 -1 bluestore(td/osd-backfill-space/13/block) _read_bdev_label failed to open td/osd-backfill-space/13/block: (2) No such file or directory 2020-09-12T14:41:38.814 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:38.814+0000 7f29212cfec0 -1 bluestore(td/osd-backfill-space/13/block) _read_bdev_label failed to open td/osd-backfill-space/13/block: (2) No such file or directory 2020-09-12T14:41:38.879 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:38.879+0000 7f29212cfec0 -1 bluestore(td/osd-backfill-space/13) _read_fsid unparsable uuid 2020-09-12T14:41:43.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/13/keyring 2020-09-12T14:41:43.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:41:43.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd13 key to auth repository 2020-09-12T14:41:43.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/13/keyring auth add osd.13 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:41:43.736 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd13 key to auth repository 2020-09-12T14:41:44.291 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.13 2020-09-12T14:41:44.292 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.13 2020-09-12T14:41:44.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:41:44.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 13 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/13 --osd-journal=td/osd-backfill-space/13/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:41:44.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:41:44.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:41:44.318 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:44.316+0000 7f6c2191dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:44.345 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:44.343+0000 7f6c2191dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:44.345 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:44.344+0000 7f6c2191dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:44.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 13 2020-09-12T14:41:44.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:41:44.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=13 2020-09-12T14:41:44.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:41:44.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:41:44.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:41:44.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:41:44.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:41:44.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.13 up' 2020-09-12T14:41:44.702 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:41:44.863 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:44.861+0000 7f6c2191dec0 -1 Falling back to public interface 2020-09-12T14:41:45.119 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:41:45.655 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:45.654+0000 7f6c2191dec0 -1 osd.13 0 log_to_monitors {default=true} 2020-09-12T14:41:46.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:41:46.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:41:46.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:41:46.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:41:46.122 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.13 up' 2020-09-12T14:41:46.122 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:41:46.608 INFO:tasks.workunit.client.0.smithi027.stdout:osd.13 up in weight 1 up_from 70 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6906/248698,v1:127.0.0.1:6907/248698] [v2:127.0.0.1:6908/248698,v1:127.0.0.1:6909/248698] exists,up 72ab1707-92b4-419e-b071-e8165a7f8af6 2020-09-12T14:41:46.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:41:46.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:41:46.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:41:46.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:199: TEST_backfill_test_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:41:46.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:201: TEST_backfill_test_multi: run_osd td/osd-backfill-space 14 2020-09-12T14:41:46.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:41:46.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:41:46.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=14 2020-09-12T14:41:46.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:41:46.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/14 2020-09-12T14:41:46.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:41:46.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:41:46.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:41:46.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:41:46.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/14' 2020-09-12T14:41:46.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/14/journal' 2020-09-12T14:41:46.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:41:46.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:41:46.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:41:46.611 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:41:46.612 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:41:46.612 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:41:46.612 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:41:46.612 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:41:46.612 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:41:46.612 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:41:46.612 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:41:46.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:41:46.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:41:46.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:41:46.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:41:46.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:41:46.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:41:46.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:41:46.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:41:46.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:41:46.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:41:46.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/14 2020-09-12T14:41:46.615 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:41:46.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=d86b1a7e-2669-49f4-a5ca-3ed4fe08c808 2020-09-12T14:41:46.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd14 d86b1a7e-2669-49f4-a5ca-3ed4fe08c808' 2020-09-12T14:41:46.615 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:41:46.615 INFO:tasks.workunit.client.0.smithi027.stdout:add osd14 d86b1a7e-2669-49f4-a5ca-3ed4fe08c808 2020-09-12T14:41:46.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQAq3lxf/PBcJRAATn/HDHaPCAM/jXDdA6SRMQ== 2020-09-12T14:41:46.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQAq3lxf/PBcJRAATn/HDHaPCAM/jXDdA6SRMQ=="}' 2020-09-12T14:41:46.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new d86b1a7e-2669-49f4-a5ca-3ed4fe08c808 -i td/osd-backfill-space/14/new.json 2020-09-12T14:41:47.458 INFO:tasks.workunit.client.0.smithi027.stdout:14 2020-09-12T14:41:47.470 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/14/new.json 2020-09-12T14:41:47.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 14 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/14 --osd-journal=td/osd-backfill-space/14/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQAq3lxf/PBcJRAATn/HDHaPCAM/jXDdA6SRMQ== --osd-uuid d86b1a7e-2669-49f4-a5ca-3ed4fe08c808 2020-09-12T14:41:47.498 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:47.497+0000 7fc1d42e8ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:47.500 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:47.499+0000 7fc1d42e8ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:47.501 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:47.500+0000 7fc1d42e8ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:47.502 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:47.501+0000 7fc1d42e8ec0 -1 bluestore(td/osd-backfill-space/14/block) _read_bdev_label failed to open td/osd-backfill-space/14/block: (2) No such file or directory 2020-09-12T14:41:47.502 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:47.501+0000 7fc1d42e8ec0 -1 bluestore(td/osd-backfill-space/14/block) _read_bdev_label failed to open td/osd-backfill-space/14/block: (2) No such file or directory 2020-09-12T14:41:47.502 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:47.501+0000 7fc1d42e8ec0 -1 bluestore(td/osd-backfill-space/14/block) _read_bdev_label failed to open td/osd-backfill-space/14/block: (2) No such file or directory 2020-09-12T14:41:47.584 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:47.583+0000 7fc1d42e8ec0 -1 bluestore(td/osd-backfill-space/14) _read_fsid unparsable uuid 2020-09-12T14:41:52.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/14/keyring 2020-09-12T14:41:52.447 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:41:52.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd14 key to auth repository 2020-09-12T14:41:52.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/14/keyring auth add osd.14 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:41:52.448 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd14 key to auth repository 2020-09-12T14:41:53.003 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.14 2020-09-12T14:41:53.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.14 2020-09-12T14:41:53.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:41:53.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 14 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/14 --osd-journal=td/osd-backfill-space/14/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:41:53.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:41:53.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:41:53.029 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:53.028+0000 7fddb403bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:53.038 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:53.037+0000 7fddb403bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:53.039 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:53.038+0000 7fddb403bec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:53.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 14 2020-09-12T14:41:53.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:41:53.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=14 2020-09-12T14:41:53.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:41:53.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:41:53.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:41:53.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:41:53.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:41:53.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.14 up' 2020-09-12T14:41:53.414 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:41:53.556 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:53.555+0000 7fddb403bec0 -1 Falling back to public interface 2020-09-12T14:41:53.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:41:54.367 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:54.366+0000 7fddb403bec0 -1 osd.14 0 log_to_monitors {default=true} 2020-09-12T14:41:54.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:41:54.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:41:54.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:41:54.846 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:41:54.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:41:54.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.14 up' 2020-09-12T14:41:55.270 INFO:tasks.workunit.client.0.smithi027.stdout:osd.14 up in weight 1 up_from 75 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6914/249692,v1:127.0.0.1:6915/249692] [v2:127.0.0.1:6916/249692,v1:127.0.0.1:6917/249692] exists,up d86b1a7e-2669-49f4-a5ca-3ed4fe08c808 2020-09-12T14:41:55.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:41:55.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:41:55.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:41:55.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:199: TEST_backfill_test_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:41:55.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:201: TEST_backfill_test_multi: run_osd td/osd-backfill-space 15 2020-09-12T14:41:55.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:41:55.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:41:55.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=15 2020-09-12T14:41:55.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:41:55.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/15 2020-09-12T14:41:55.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:41:55.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:41:55.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:41:55.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:41:55.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/15' 2020-09-12T14:41:55.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/15/journal' 2020-09-12T14:41:55.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:41:55.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:41:55.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:41:55.273 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:41:55.273 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:41:55.273 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:41:55.273 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:41:55.273 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:41:55.273 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:41:55.274 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:41:55.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:41:55.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:41:55.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:41:55.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:41:55.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:41:55.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:41:55.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:41:55.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:41:55.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:41:55.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:41:55.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:41:55.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/15 2020-09-12T14:41:55.276 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:41:55.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=a86745b0-fc17-4885-96ec-1a371eb6e205 2020-09-12T14:41:55.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd15 a86745b0-fc17-4885-96ec-1a371eb6e205' 2020-09-12T14:41:55.276 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:41:55.277 INFO:tasks.workunit.client.0.smithi027.stdout:add osd15 a86745b0-fc17-4885-96ec-1a371eb6e205 2020-09-12T14:41:55.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQAz3lxf8H8vERAA5NNJGuA1WLHMrG80k0pzaw== 2020-09-12T14:41:55.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQAz3lxf8H8vERAA5NNJGuA1WLHMrG80k0pzaw=="}' 2020-09-12T14:41:55.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new a86745b0-fc17-4885-96ec-1a371eb6e205 -i td/osd-backfill-space/15/new.json 2020-09-12T14:41:55.853 INFO:tasks.workunit.client.0.smithi027.stdout:15 2020-09-12T14:41:55.864 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/15/new.json 2020-09-12T14:41:55.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 15 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/15 --osd-journal=td/osd-backfill-space/15/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQAz3lxf8H8vERAA5NNJGuA1WLHMrG80k0pzaw== --osd-uuid a86745b0-fc17-4885-96ec-1a371eb6e205 2020-09-12T14:41:55.893 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:55.892+0000 7f71ba521ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:55.901 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:55.900+0000 7f71ba521ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:55.902 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:55.901+0000 7f71ba521ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:41:55.903 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:55.902+0000 7f71ba521ec0 -1 bluestore(td/osd-backfill-space/15/block) _read_bdev_label failed to open td/osd-backfill-space/15/block: (2) No such file or directory 2020-09-12T14:41:55.903 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:55.902+0000 7f71ba521ec0 -1 bluestore(td/osd-backfill-space/15/block) _read_bdev_label failed to open td/osd-backfill-space/15/block: (2) No such file or directory 2020-09-12T14:41:55.903 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:55.902+0000 7f71ba521ec0 -1 bluestore(td/osd-backfill-space/15/block) _read_bdev_label failed to open td/osd-backfill-space/15/block: (2) No such file or directory 2020-09-12T14:41:55.971 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:41:55.968+0000 7f71ba521ec0 -1 bluestore(td/osd-backfill-space/15) _read_fsid unparsable uuid 2020-09-12T14:42:00.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/15/keyring 2020-09-12T14:42:00.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:42:00.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd15 key to auth repository 2020-09-12T14:42:00.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/15/keyring auth add osd.15 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:42:00.742 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd15 key to auth repository 2020-09-12T14:42:01.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.15 2020-09-12T14:42:01.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 15 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/15 --osd-journal=td/osd-backfill-space/15/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:42:01.288 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.15 2020-09-12T14:42:01.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:42:01.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:42:01.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:42:01.313 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:01.313+0000 7f8a00198ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:01.333 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:01.332+0000 7f8a00198ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:01.334 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:01.334+0000 7f8a00198ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:01.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 15 2020-09-12T14:42:01.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:42:01.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=15 2020-09-12T14:42:01.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:42:01.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:42:01.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:42:01.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:42:01.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:42:01.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.15 up' 2020-09-12T14:42:01.719 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:42:01.855 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:01.854+0000 7f8a00198ec0 -1 Falling back to public interface 2020-09-12T14:42:02.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:42:02.744 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:02.744+0000 7f8a00198ec0 -1 osd.15 0 log_to_monitors {default=true} 2020-09-12T14:42:03.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:42:03.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:42:03.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:42:03.132 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:42:03.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:42:03.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.15 up' 2020-09-12T14:42:03.566 INFO:tasks.workunit.client.0.smithi027.stdout:osd.15 up in weight 1 up_from 80 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6922/250694,v1:127.0.0.1:6923/250694] [v2:127.0.0.1:6924/250694,v1:127.0.0.1:6925/250694] exists,up a86745b0-fc17-4885-96ec-1a371eb6e205 2020-09-12T14:42:03.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:42:03.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:42:03.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:42:03.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:199: TEST_backfill_test_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:42:03.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:201: TEST_backfill_test_multi: run_osd td/osd-backfill-space 16 2020-09-12T14:42:03.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:42:03.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:42:03.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=16 2020-09-12T14:42:03.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:42:03.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/16 2020-09-12T14:42:03.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:42:03.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:42:03.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:42:03.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:42:03.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/16' 2020-09-12T14:42:03.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/16/journal' 2020-09-12T14:42:03.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:42:03.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:42:03.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:42:03.569 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:42:03.569 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:42:03.569 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:42:03.569 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:42:03.569 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:42:03.569 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:42:03.570 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:42:03.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:42:03.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:42:03.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:42:03.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:42:03.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:42:03.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:42:03.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:42:03.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:42:03.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:42:03.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:42:03.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:42:03.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/16 2020-09-12T14:42:03.572 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:42:03.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=a644f596-47b1-4079-9153-588369f651d2 2020-09-12T14:42:03.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd16 a644f596-47b1-4079-9153-588369f651d2' 2020-09-12T14:42:03.572 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:42:03.573 INFO:tasks.workunit.client.0.smithi027.stdout:add osd16 a644f596-47b1-4079-9153-588369f651d2 2020-09-12T14:42:03.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQA73lxfjQLbIhAAV45EArkuRk6tbFRoiWtJ5Q== 2020-09-12T14:42:03.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQA73lxfjQLbIhAAV45EArkuRk6tbFRoiWtJ5Q=="}' 2020-09-12T14:42:03.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new a644f596-47b1-4079-9153-588369f651d2 -i td/osd-backfill-space/16/new.json 2020-09-12T14:42:04.256 INFO:tasks.workunit.client.0.smithi027.stdout:16 2020-09-12T14:42:04.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/16/new.json 2020-09-12T14:42:04.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 16 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/16 --osd-journal=td/osd-backfill-space/16/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQA73lxfjQLbIhAAV45EArkuRk6tbFRoiWtJ5Q== --osd-uuid a644f596-47b1-4079-9153-588369f651d2 2020-09-12T14:42:04.296 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:04.295+0000 7f3dbd54dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:04.297 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:04.297+0000 7f3dbd54dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:04.298 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:04.298+0000 7f3dbd54dec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:04.299 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:04.299+0000 7f3dbd54dec0 -1 bluestore(td/osd-backfill-space/16/block) _read_bdev_label failed to open td/osd-backfill-space/16/block: (2) No such file or directory 2020-09-12T14:42:04.299 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:04.299+0000 7f3dbd54dec0 -1 bluestore(td/osd-backfill-space/16/block) _read_bdev_label failed to open td/osd-backfill-space/16/block: (2) No such file or directory 2020-09-12T14:42:04.299 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:04.299+0000 7f3dbd54dec0 -1 bluestore(td/osd-backfill-space/16/block) _read_bdev_label failed to open td/osd-backfill-space/16/block: (2) No such file or directory 2020-09-12T14:42:04.347 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:04.346+0000 7f3dbd54dec0 -1 bluestore(td/osd-backfill-space/16) _read_fsid unparsable uuid 2020-09-12T14:42:09.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/16/keyring 2020-09-12T14:42:09.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:42:09.145 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd16 key to auth repository 2020-09-12T14:42:09.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd16 key to auth repository 2020-09-12T14:42:09.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/16/keyring auth add osd.16 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:42:09.699 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.16 2020-09-12T14:42:09.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.16 2020-09-12T14:42:09.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:42:09.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 16 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/16 --osd-journal=td/osd-backfill-space/16/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:42:09.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:42:09.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:42:09.725 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:09.725+0000 7fa198a33ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:09.736 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:09.736+0000 7fa198a33ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:09.737 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:09.737+0000 7fa198a33ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:10.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 16 2020-09-12T14:42:10.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:42:10.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=16 2020-09-12T14:42:10.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:42:10.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:42:10.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:42:10.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:42:10.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:42:10.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.16 up' 2020-09-12T14:42:10.109 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:42:10.254 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:10.253+0000 7fa198a33ec0 -1 Falling back to public interface 2020-09-12T14:42:10.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:42:11.089 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:11.089+0000 7fa198a33ec0 -1 osd.16 0 log_to_monitors {default=true} 2020-09-12T14:42:11.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:42:11.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:42:11.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:42:11.517 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:42:11.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:42:11.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.16 up' 2020-09-12T14:42:11.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:42:12.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:42:12.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:42:12.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 2 2020-09-12T14:42:12.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:42:12.941 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:42:12.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.16 up' 2020-09-12T14:42:13.359 INFO:tasks.workunit.client.0.smithi027.stdout:osd.16 up in weight 1 up_from 85 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6930/251690,v1:127.0.0.1:6931/251690] [v2:127.0.0.1:6932/251690,v1:127.0.0.1:6933/251690] exists,up a644f596-47b1-4079-9153-588369f651d2 2020-09-12T14:42:13.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:42:13.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:42:13.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:42:13.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:199: TEST_backfill_test_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:42:13.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:201: TEST_backfill_test_multi: run_osd td/osd-backfill-space 17 2020-09-12T14:42:13.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:42:13.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:42:13.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=17 2020-09-12T14:42:13.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:42:13.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/17 2020-09-12T14:42:13.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:42:13.362 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:42:13.362 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:42:13.362 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:42:13.362 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/17' 2020-09-12T14:42:13.362 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/17/journal' 2020-09-12T14:42:13.362 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:42:13.362 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:42:13.363 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:42:13.363 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:42:13.363 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:42:13.363 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:42:13.363 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:42:13.363 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:42:13.363 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:42:13.363 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:42:13.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:42:13.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:42:13.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:42:13.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:42:13.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:42:13.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:42:13.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:42:13.365 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:42:13.365 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:42:13.365 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:42:13.365 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:42:13.365 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/17 2020-09-12T14:42:13.366 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:42:13.366 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=d431658b-b6c4-447d-bbbf-a2fc72400991 2020-09-12T14:42:13.366 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd17 d431658b-b6c4-447d-bbbf-a2fc72400991' 2020-09-12T14:42:13.366 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:42:13.366 INFO:tasks.workunit.client.0.smithi027.stdout:add osd17 d431658b-b6c4-447d-bbbf-a2fc72400991 2020-09-12T14:42:13.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQBF3lxf+86UFhAAMyQN9dGSeBcloH8mtvZmow== 2020-09-12T14:42:13.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQBF3lxf+86UFhAAMyQN9dGSeBcloH8mtvZmow=="}' 2020-09-12T14:42:13.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new d431658b-b6c4-447d-bbbf-a2fc72400991 -i td/osd-backfill-space/17/new.json 2020-09-12T14:42:13.919 INFO:tasks.workunit.client.0.smithi027.stdout:17 2020-09-12T14:42:13.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/17/new.json 2020-09-12T14:42:13.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 17 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/17 --osd-journal=td/osd-backfill-space/17/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQBF3lxf+86UFhAAMyQN9dGSeBcloH8mtvZmow== --osd-uuid d431658b-b6c4-447d-bbbf-a2fc72400991 2020-09-12T14:42:13.959 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:13.959+0000 7fbb0f170ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:13.960 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:13.960+0000 7fbb0f170ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:13.962 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:13.962+0000 7fbb0f170ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:13.963 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:13.963+0000 7fbb0f170ec0 -1 bluestore(td/osd-backfill-space/17/block) _read_bdev_label failed to open td/osd-backfill-space/17/block: (2) No such file or directory 2020-09-12T14:42:13.963 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:13.963+0000 7fbb0f170ec0 -1 bluestore(td/osd-backfill-space/17/block) _read_bdev_label failed to open td/osd-backfill-space/17/block: (2) No such file or directory 2020-09-12T14:42:13.963 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:13.963+0000 7fbb0f170ec0 -1 bluestore(td/osd-backfill-space/17/block) _read_bdev_label failed to open td/osd-backfill-space/17/block: (2) No such file or directory 2020-09-12T14:42:14.062 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:14.062+0000 7fbb0f170ec0 -1 bluestore(td/osd-backfill-space/17) _read_fsid unparsable uuid 2020-09-12T14:42:18.944 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/17/keyring 2020-09-12T14:42:18.944 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:42:18.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd17 key to auth repository 2020-09-12T14:42:18.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/17/keyring auth add osd.17 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:42:18.945 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd17 key to auth repository 2020-09-12T14:42:19.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.17 2020-09-12T14:42:19.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 17 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/17 --osd-journal=td/osd-backfill-space/17/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:42:19.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:42:19.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:42:19.508 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:42:19.508 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.17 2020-09-12T14:42:19.531 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:19.530+0000 7f6de02b7ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:19.533 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:19.532+0000 7f6de02b7ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:19.535 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:19.534+0000 7f6de02b7ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:19.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 17 2020-09-12T14:42:19.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:42:19.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=17 2020-09-12T14:42:19.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:42:19.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:42:19.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:42:19.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:42:19.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:42:19.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.17 up' 2020-09-12T14:42:19.917 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:42:20.053 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:20.052+0000 7f6de02b7ec0 -1 Falling back to public interface 2020-09-12T14:42:20.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:42:20.913 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:20.912+0000 7f6de02b7ec0 -1 osd.17 0 log_to_monitors {default=true} 2020-09-12T14:42:21.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:42:21.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:42:21.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:42:21.325 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:42:21.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.17 up' 2020-09-12T14:42:21.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:42:21.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:42:22.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:42:22.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:42:22.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 2 2020-09-12T14:42:22.833 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:42:22.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:42:22.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.17 up' 2020-09-12T14:42:23.269 INFO:tasks.workunit.client.0.smithi027.stdout:osd.17 up in weight 1 up_from 90 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6938/252710,v1:127.0.0.1:6939/252710] [v2:127.0.0.1:6940/252710,v1:127.0.0.1:6941/252710] exists,up d431658b-b6c4-447d-bbbf-a2fc72400991 2020-09-12T14:42:23.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:42:23.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:42:23.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:42:23.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:199: TEST_backfill_test_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:42:23.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:201: TEST_backfill_test_multi: run_osd td/osd-backfill-space 18 2020-09-12T14:42:23.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:42:23.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:42:23.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=18 2020-09-12T14:42:23.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:42:23.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/18 2020-09-12T14:42:23.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:42:23.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:42:23.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:42:23.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:42:23.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/18' 2020-09-12T14:42:23.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/18/journal' 2020-09-12T14:42:23.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:42:23.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:42:23.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:42:23.274 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:42:23.274 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:42:23.275 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:42:23.275 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:42:23.275 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:42:23.275 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:42:23.275 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:42:23.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:42:23.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:42:23.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:42:23.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:42:23.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:42:23.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:42:23.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:42:23.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:42:23.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:42:23.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:42:23.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:42:23.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/18 2020-09-12T14:42:23.278 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:42:23.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=f9f9fe35-b9c8-40e8-bf55-18975906df99 2020-09-12T14:42:23.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd18 f9f9fe35-b9c8-40e8-bf55-18975906df99' 2020-09-12T14:42:23.279 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:42:23.280 INFO:tasks.workunit.client.0.smithi027.stdout:add osd18 f9f9fe35-b9c8-40e8-bf55-18975906df99 2020-09-12T14:42:23.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQBP3lxf/TcZERAADZCfTtxikZZhgp/xmvdjuQ== 2020-09-12T14:42:23.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQBP3lxf/TcZERAADZCfTtxikZZhgp/xmvdjuQ=="}' 2020-09-12T14:42:23.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new f9f9fe35-b9c8-40e8-bf55-18975906df99 -i td/osd-backfill-space/18/new.json 2020-09-12T14:42:23.799 INFO:tasks.workunit.client.0.smithi027.stdout:18 2020-09-12T14:42:23.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/18/new.json 2020-09-12T14:42:23.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 18 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/18 --osd-journal=td/osd-backfill-space/18/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQBP3lxf/TcZERAADZCfTtxikZZhgp/xmvdjuQ== --osd-uuid f9f9fe35-b9c8-40e8-bf55-18975906df99 2020-09-12T14:42:23.838 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:23.837+0000 7fb9530f1ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:23.840 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:23.839+0000 7fb9530f1ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:23.841 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:23.840+0000 7fb9530f1ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:23.842 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:23.841+0000 7fb9530f1ec0 -1 bluestore(td/osd-backfill-space/18/block) _read_bdev_label failed to open td/osd-backfill-space/18/block: (2) No such file or directory 2020-09-12T14:42:23.842 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:23.841+0000 7fb9530f1ec0 -1 bluestore(td/osd-backfill-space/18/block) _read_bdev_label failed to open td/osd-backfill-space/18/block: (2) No such file or directory 2020-09-12T14:42:23.842 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:23.841+0000 7fb9530f1ec0 -1 bluestore(td/osd-backfill-space/18/block) _read_bdev_label failed to open td/osd-backfill-space/18/block: (2) No such file or directory 2020-09-12T14:42:23.965 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:23.964+0000 7fb9530f1ec0 -1 bluestore(td/osd-backfill-space/18) _read_fsid unparsable uuid 2020-09-12T14:42:28.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/18/keyring 2020-09-12T14:42:28.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:42:28.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd18 key to auth repository 2020-09-12T14:42:28.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/18/keyring auth add osd.18 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:42:28.805 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd18 key to auth repository 2020-09-12T14:42:29.359 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.18 2020-09-12T14:42:29.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.18 2020-09-12T14:42:29.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 18 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/18 --osd-journal=td/osd-backfill-space/18/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:42:29.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:42:29.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:42:29.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:42:29.385 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:29.384+0000 7f2ca505fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:29.387 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:29.386+0000 7f2ca505fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:29.389 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:29.388+0000 7f2ca505fec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:29.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 18 2020-09-12T14:42:29.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:42:29.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=18 2020-09-12T14:42:29.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:42:29.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:42:29.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:42:29.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:42:29.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:42:29.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.18 up' 2020-09-12T14:42:29.774 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:42:29.909 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:29.908+0000 7f2ca505fec0 -1 Falling back to public interface 2020-09-12T14:42:30.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:42:30.714 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:30.713+0000 7f2ca505fec0 -1 osd.18 0 log_to_monitors {default=true} 2020-09-12T14:42:31.196 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:42:31.197 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:42:31.197 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:42:31.197 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:42:31.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:42:31.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.18 up' 2020-09-12T14:42:31.617 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:42:32.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:42:32.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:42:32.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 2 2020-09-12T14:42:32.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:42:32.620 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:42:32.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.18 up' 2020-09-12T14:42:33.059 INFO:tasks.workunit.client.0.smithi027.stdout:osd.18 up in weight 1 up_from 95 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6946/253731,v1:127.0.0.1:6947/253731] [v2:127.0.0.1:6948/253731,v1:127.0.0.1:6949/253731] exists,up f9f9fe35-b9c8-40e8-bf55-18975906df99 2020-09-12T14:42:33.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:42:33.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:42:33.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:42:33.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:199: TEST_backfill_test_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2020-09-12T14:42:33.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:201: TEST_backfill_test_multi: run_osd td/osd-backfill-space 19 2020-09-12T14:42:33.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: local dir=td/osd-backfill-space 2020-09-12T14:42:33.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: shift 2020-09-12T14:42:33.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: local id=19 2020-09-12T14:42:33.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: shift 2020-09-12T14:42:33.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: local osd_data=td/osd-backfill-space/19 2020-09-12T14:42:33.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: local 'ceph_args=--fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 ' 2020-09-12T14:42:33.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2020-09-12T14:42:33.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-journal-size=100' 2020-09-12T14:42:33.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2020-09-12T14:42:33.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-data=td/osd-backfill-space/19' 2020-09-12T14:42:33.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-journal=td/osd-backfill-space/19/journal' 2020-09-12T14:42:33.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --chdir=' 2020-09-12T14:42:33.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+= 2020-09-12T14:42:33.062 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --run-dir=td/osd-backfill-space' 2020-09-12T14:42:33.062 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: get_asok_path 2020-09-12T14:42:33.062 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2020-09-12T14:42:33.062 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2020-09-12T14:42:33.062 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2020-09-12T14:42:33.062 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2020-09-12T14:42:33.062 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.145692 2020-09-12T14:42:33.063 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:42:33.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' 2020-09-12T14:42:33.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --debug-osd=20' 2020-09-12T14:42:33.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-ms=1' 2020-09-12T14:42:33.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-monc=20' 2020-09-12T14:42:33.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --log-file=td/osd-backfill-space/$name.log' 2020-09-12T14:42:33.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --pid-file=td/osd-backfill-space/$name.pid' 2020-09-12T14:42:33.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2020-09-12T14:42:33.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2020-09-12T14:42:33.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:42:33.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' ' 2020-09-12T14:42:33.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+= 2020-09-12T14:42:33.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: mkdir -p td/osd-backfill-space/19 2020-09-12T14:42:33.065 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: uuidgen 2020-09-12T14:42:33.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: local uuid=84c07458-ca1d-4713-9dc5-a5461cabb9f0 2020-09-12T14:42:33.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: echo 'add osd19 84c07458-ca1d-4713-9dc5-a5461cabb9f0' 2020-09-12T14:42:33.065 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: ceph-authtool --gen-print-key 2020-09-12T14:42:33.066 INFO:tasks.workunit.client.0.smithi027.stdout:add osd19 84c07458-ca1d-4713-9dc5-a5461cabb9f0 2020-09-12T14:42:33.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: OSD_SECRET=AQBZ3lxfB7ecBBAAUDtzYQaT2q4F4Ku9QTz1bg== 2020-09-12T14:42:33.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo '{"cephx_secret": "AQBZ3lxfB7ecBBAAUDtzYQaT2q4F4Ku9QTz1bg=="}' 2020-09-12T14:42:33.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph osd new 84c07458-ca1d-4713-9dc5-a5461cabb9f0 -i td/osd-backfill-space/19/new.json 2020-09-12T14:42:33.658 INFO:tasks.workunit.client.0.smithi027.stdout:19 2020-09-12T14:42:33.669 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: rm td/osd-backfill-space/19/new.json 2020-09-12T14:42:33.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph-osd -i 19 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/19 --osd-journal=td/osd-backfill-space/19/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQBZ3lxfB7ecBBAAUDtzYQaT2q4F4Ku9QTz1bg== --osd-uuid 84c07458-ca1d-4713-9dc5-a5461cabb9f0 2020-09-12T14:42:33.698 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:33.697+0000 7fe44b5dbec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:33.699 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:33.698+0000 7fe44b5dbec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:33.701 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:33.700+0000 7fe44b5dbec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:33.702 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:33.700+0000 7fe44b5dbec0 -1 bluestore(td/osd-backfill-space/19/block) _read_bdev_label failed to open td/osd-backfill-space/19/block: (2) No such file or directory 2020-09-12T14:42:33.702 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:33.700+0000 7fe44b5dbec0 -1 bluestore(td/osd-backfill-space/19/block) _read_bdev_label failed to open td/osd-backfill-space/19/block: (2) No such file or directory 2020-09-12T14:42:33.702 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:33.700+0000 7fe44b5dbec0 -1 bluestore(td/osd-backfill-space/19/block) _read_bdev_label failed to open td/osd-backfill-space/19/block: (2) No such file or directory 2020-09-12T14:42:33.922 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:33.922+0000 7fe44b5dbec0 -1 bluestore(td/osd-backfill-space/19) _read_fsid unparsable uuid 2020-09-12T14:42:38.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: local key_fn=td/osd-backfill-space/19/keyring 2020-09-12T14:42:38.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: cat 2020-09-12T14:42:38.889 INFO:tasks.workunit.client.0.smithi027.stdout:adding osd19 key to auth repository 2020-09-12T14:42:38.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:673: run_osd: echo adding osd19 key to auth repository 2020-09-12T14:42:38.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: ceph -i td/osd-backfill-space/19/keyring auth add osd.19 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2020-09-12T14:42:39.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo start osd.19 2020-09-12T14:42:39.447 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph-osd -i 19 --fsid=0a07d34a-ab83-4d05-8c3d-78eb24f9be2f --auth-supported=none --mon-host=127.0.0.1:7180 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --fake_statfs_for_testing=3686400 --osd_max_backfills=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-backfill-space/19 --osd-journal=td/osd-backfill-space/19/journal --chdir= --run-dir=td/osd-backfill-space '--admin-socket=/tmp/ceph-asok.145692/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-backfill-space/$name.log' '--pid-file=td/osd-backfill-space/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2020-09-12T14:42:39.447 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: ceph osd dump --format=json 2020-09-12T14:42:39.447 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: jq '.flags_set[]' 2020-09-12T14:42:39.447 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:679: run_osd: grep -q '"noup"' 2020-09-12T14:42:39.447 INFO:tasks.workunit.client.0.smithi027.stdout:start osd.19 2020-09-12T14:42:39.472 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:39.472+0000 7f4a1b828ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:39.480 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:39.480+0000 7f4a1b828ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:39.482 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:39.482+0000 7f4a1b828ec0 -1 WARNING: all dangerous and experimental features are enabled. 2020-09-12T14:42:39.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:682: run_osd: wait_for_osd up 19 2020-09-12T14:42:39.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: wait_for_osd: local state=up 2020-09-12T14:42:39.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:917: wait_for_osd: local id=19 2020-09-12T14:42:39.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: wait_for_osd: status=1 2020-09-12T14:42:39.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i=0 )) 2020-09-12T14:42:39.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:42:39.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 0 2020-09-12T14:42:39.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:42:39.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.19 up' 2020-09-12T14:42:39.861 INFO:tasks.workunit.client.0.smithi027.stdout:0 2020-09-12T14:42:39.999 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:39.997+0000 7f4a1b828ec0 -1 Falling back to public interface 2020-09-12T14:42:40.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:42:40.816 INFO:tasks.workunit.client.0.smithi027.stderr:2020-09-12T14:42:40.815+0000 7f4a1b828ec0 -1 osd.19 0 log_to_monitors {default=true} 2020-09-12T14:42:41.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:42:41.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:42:41.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 1 2020-09-12T14:42:41.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:42:41.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.19 up' 2020-09-12T14:42:41.272 INFO:tasks.workunit.client.0.smithi027.stdout:1 2020-09-12T14:42:41.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:923: wait_for_osd: sleep 1 2020-09-12T14:42:42.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i++ )) 2020-09-12T14:42:42.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:920: wait_for_osd: (( i < 300 )) 2020-09-12T14:42:42.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: wait_for_osd: echo 2 2020-09-12T14:42:42.705 INFO:tasks.workunit.client.0.smithi027.stdout:2 2020-09-12T14:42:42.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: ceph osd dump 2020-09-12T14:42:42.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:922: wait_for_osd: grep 'osd.19 up' 2020-09-12T14:42:43.160 INFO:tasks.workunit.client.0.smithi027.stdout:osd.19 up in weight 1 up_from 100 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6954/254751,v1:127.0.0.1:6955/254751] [v2:127.0.0.1:6956/254751,v1:127.0.0.1:6957/254751] exists,up 84c07458-ca1d-4713-9dc5-a5461cabb9f0 2020-09-12T14:42:43.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: wait_for_osd: status=0 2020-09-12T14:42:43.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:926: wait_for_osd: break 2020-09-12T14:42:43.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: wait_for_osd: return 0 2020-09-12T14:42:43.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:204: TEST_backfill_test_multi: ceph osd set-backfillfull-ratio .85 2020-09-12T14:42:43.881 INFO:tasks.workunit.client.0.smithi027.stderr:osd set-backfillfull-ratio 0.85 2020-09-12T14:42:43.891 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:206: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:42:43.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:206: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:42:43.892 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:208: TEST_backfill_test_multi: create_pool test1 1 1 2020-09-12T14:42:43.892 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test1 1 1 2020-09-12T14:42:44.696 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test1' already exists 2020-09-12T14:42:44.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:42:45.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:209: TEST_backfill_test_multi: ceph osd pool set test1 size 1 --yes-i-really-mean-it 2020-09-12T14:42:46.762 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 1 size to 1 2020-09-12T14:42:46.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:206: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:42:46.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:208: TEST_backfill_test_multi: create_pool test2 1 1 2020-09-12T14:42:46.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test2 1 1 2020-09-12T14:42:47.610 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test2' already exists 2020-09-12T14:42:47.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:42:48.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:209: TEST_backfill_test_multi: ceph osd pool set test2 size 1 --yes-i-really-mean-it 2020-09-12T14:42:49.944 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 2 size to 1 2020-09-12T14:42:49.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:206: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:42:49.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:208: TEST_backfill_test_multi: create_pool test3 1 1 2020-09-12T14:42:49.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test3 1 1 2020-09-12T14:42:50.587 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test3' already exists 2020-09-12T14:42:50.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:42:51.598 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:209: TEST_backfill_test_multi: ceph osd pool set test3 size 1 --yes-i-really-mean-it 2020-09-12T14:42:52.622 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 3 size to 1 2020-09-12T14:42:52.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:206: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:42:52.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:208: TEST_backfill_test_multi: create_pool test4 1 1 2020-09-12T14:42:52.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test4 1 1 2020-09-12T14:42:53.464 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test4' already exists 2020-09-12T14:42:53.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:42:54.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:209: TEST_backfill_test_multi: ceph osd pool set test4 size 1 --yes-i-really-mean-it 2020-09-12T14:42:55.493 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 4 size to 1 2020-09-12T14:42:55.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:206: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:42:55.503 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:208: TEST_backfill_test_multi: create_pool test5 1 1 2020-09-12T14:42:55.504 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test5 1 1 2020-09-12T14:42:56.460 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test5' already exists 2020-09-12T14:42:56.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:42:57.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:209: TEST_backfill_test_multi: ceph osd pool set test5 size 1 --yes-i-really-mean-it 2020-09-12T14:42:58.453 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 5 size to 1 2020-09-12T14:42:58.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:206: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:42:58.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:208: TEST_backfill_test_multi: create_pool test6 1 1 2020-09-12T14:42:58.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test6 1 1 2020-09-12T14:42:59.301 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test6' already exists 2020-09-12T14:42:59.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:43:00.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:209: TEST_backfill_test_multi: ceph osd pool set test6 size 1 --yes-i-really-mean-it 2020-09-12T14:43:01.246 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 6 size to 1 2020-09-12T14:43:01.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:206: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:01.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:208: TEST_backfill_test_multi: create_pool test7 1 1 2020-09-12T14:43:01.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test7 1 1 2020-09-12T14:43:01.880 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test7' already exists 2020-09-12T14:43:01.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:43:02.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:209: TEST_backfill_test_multi: ceph osd pool set test7 size 1 --yes-i-really-mean-it 2020-09-12T14:43:03.893 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 7 size to 1 2020-09-12T14:43:03.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:206: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:03.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:208: TEST_backfill_test_multi: create_pool test8 1 1 2020-09-12T14:43:03.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:540: create_pool: ceph osd pool create test8 1 1 2020-09-12T14:43:04.686 INFO:tasks.workunit.client.0.smithi027.stderr:pool 'test8' already exists 2020-09-12T14:43:04.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: sleep 1 2020-09-12T14:43:05.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:209: TEST_backfill_test_multi: ceph osd pool set test8 size 1 --yes-i-really-mean-it 2020-09-12T14:43:06.792 INFO:tasks.workunit.client.0.smithi027.stderr:set pool 8 size to 1 2020-09-12T14:43:06.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:212: TEST_backfill_test_multi: wait_for_clean 2020-09-12T14:43:06.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: wait_for_clean: local cmd= 2020-09-12T14:43:06.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: wait_for_clean: local num_active_clean=-1 2020-09-12T14:43:06.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: wait_for_clean: local cur_active_clean 2020-09-12T14:43:06.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: delays=($(get_timeout_delays $WAIT_FOR_CLEAN_TIMEOUT .1)) 2020-09-12T14:43:06.803 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: get_timeout_delays 90 .1 2020-09-12T14:43:06.803 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: shopt -q -o xtrace 2020-09-12T14:43:06.803 INFO:tasks.workunit.client.0.smithi027.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: echo true 2020-09-12T14:43:06.803 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_timeout_delays: local trace=true 2020-09-12T14:43:06.804 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: true 2020-09-12T14:43:06.804 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_timeout_delays: shopt -u -o xtrace 2020-09-12T14:43:06.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: wait_for_clean: local -a delays 2020-09-12T14:43:06.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: wait_for_clean: local -i loop=0 2020-09-12T14:43:06.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1586: wait_for_clean: flush_pg_stats 2020-09-12T14:43:06.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: flush_pg_stats: local timeout=300 2020-09-12T14:43:06.900 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ceph osd ls 2020-09-12T14:43:07.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: flush_pg_stats: ids='0 2020-09-12T14:43:07.335 INFO:tasks.workunit.client.0.smithi027.stderr:1 2020-09-12T14:43:07.335 INFO:tasks.workunit.client.0.smithi027.stderr:2 2020-09-12T14:43:07.335 INFO:tasks.workunit.client.0.smithi027.stderr:3 2020-09-12T14:43:07.335 INFO:tasks.workunit.client.0.smithi027.stderr:4 2020-09-12T14:43:07.336 INFO:tasks.workunit.client.0.smithi027.stderr:5 2020-09-12T14:43:07.336 INFO:tasks.workunit.client.0.smithi027.stderr:6 2020-09-12T14:43:07.336 INFO:tasks.workunit.client.0.smithi027.stderr:7 2020-09-12T14:43:07.336 INFO:tasks.workunit.client.0.smithi027.stderr:8 2020-09-12T14:43:07.336 INFO:tasks.workunit.client.0.smithi027.stderr:9 2020-09-12T14:43:07.336 INFO:tasks.workunit.client.0.smithi027.stderr:10 2020-09-12T14:43:07.336 INFO:tasks.workunit.client.0.smithi027.stderr:11 2020-09-12T14:43:07.336 INFO:tasks.workunit.client.0.smithi027.stderr:12 2020-09-12T14:43:07.337 INFO:tasks.workunit.client.0.smithi027.stderr:13 2020-09-12T14:43:07.337 INFO:tasks.workunit.client.0.smithi027.stderr:14 2020-09-12T14:43:07.337 INFO:tasks.workunit.client.0.smithi027.stderr:15 2020-09-12T14:43:07.337 INFO:tasks.workunit.client.0.smithi027.stderr:16 2020-09-12T14:43:07.337 INFO:tasks.workunit.client.0.smithi027.stderr:17 2020-09-12T14:43:07.337 INFO:tasks.workunit.client.0.smithi027.stderr:18 2020-09-12T14:43:07.337 INFO:tasks.workunit.client.0.smithi027.stderr:19' 2020-09-12T14:43:07.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2070: flush_pg_stats: seqs= 2020-09-12T14:43:07.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:43:07.338 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2020-09-12T14:43:07.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=21474836519 2020-09-12T14:43:07.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 21474836519 2020-09-12T14:43:07.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836519' 2020-09-12T14:43:07.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:43:07.494 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2020-09-12T14:43:07.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=42949672998 2020-09-12T14:43:07.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 42949672998 2020-09-12T14:43:07.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836519 1-42949672998' 2020-09-12T14:43:07.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:43:07.639 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2020-09-12T14:43:07.841 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=64424509477 2020-09-12T14:43:07.841 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 64424509477 2020-09-12T14:43:07.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509477' 2020-09-12T14:43:07.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:43:07.842 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2020-09-12T14:43:07.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=85899345956 2020-09-12T14:43:07.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 85899345956 2020-09-12T14:43:07.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509477 3-85899345956' 2020-09-12T14:43:07.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:43:07.986 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2020-09-12T14:43:08.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=107374182434 2020-09-12T14:43:08.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 107374182434 2020-09-12T14:43:08.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509477 3-85899345956 4-107374182434' 2020-09-12T14:43:08.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:43:08.131 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2020-09-12T14:43:08.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=128849018912 2020-09-12T14:43:08.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 128849018912 2020-09-12T14:43:08.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509477 3-85899345956 4-107374182434 5-128849018912' 2020-09-12T14:43:08.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:43:08.277 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2020-09-12T14:43:08.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=150323855390 2020-09-12T14:43:08.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 150323855390 2020-09-12T14:43:08.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509477 3-85899345956 4-107374182434 5-128849018912 6-150323855390' 2020-09-12T14:43:08.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:43:08.421 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2020-09-12T14:43:08.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=171798691869 2020-09-12T14:43:08.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 171798691869 2020-09-12T14:43:08.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509477 3-85899345956 4-107374182434 5-128849018912 6-150323855390 7-171798691869' 2020-09-12T14:43:08.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:43:08.567 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2020-09-12T14:43:08.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=193273528347 2020-09-12T14:43:08.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 193273528347 2020-09-12T14:43:08.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509477 3-85899345956 4-107374182434 5-128849018912 6-150323855390 7-171798691869 8-193273528347' 2020-09-12T14:43:08.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:43:08.713 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2020-09-12T14:43:08.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=214748364825 2020-09-12T14:43:08.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 214748364825 2020-09-12T14:43:08.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509477 3-85899345956 4-107374182434 5-128849018912 6-150323855390 7-171798691869 8-193273528347 9-214748364825' 2020-09-12T14:43:08.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:43:08.857 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2020-09-12T14:43:09.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=236223201304 2020-09-12T14:43:09.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 236223201304 2020-09-12T14:43:09.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509477 3-85899345956 4-107374182434 5-128849018912 6-150323855390 7-171798691869 8-193273528347 9-214748364825 10-236223201304' 2020-09-12T14:43:09.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:43:09.003 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.11 flush_pg_stats 2020-09-12T14:43:09.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=257698037782 2020-09-12T14:43:09.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 257698037782 2020-09-12T14:43:09.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509477 3-85899345956 4-107374182434 5-128849018912 6-150323855390 7-171798691869 8-193273528347 9-214748364825 10-236223201304 11-257698037782' 2020-09-12T14:43:09.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:43:09.146 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.12 flush_pg_stats 2020-09-12T14:43:09.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=279172874261 2020-09-12T14:43:09.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 279172874261 2020-09-12T14:43:09.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509477 3-85899345956 4-107374182434 5-128849018912 6-150323855390 7-171798691869 8-193273528347 9-214748364825 10-236223201304 11-257698037782 12-279172874261' 2020-09-12T14:43:09.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:43:09.289 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.13 flush_pg_stats 2020-09-12T14:43:09.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=300647710739 2020-09-12T14:43:09.433 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 300647710739 2020-09-12T14:43:09.433 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509477 3-85899345956 4-107374182434 5-128849018912 6-150323855390 7-171798691869 8-193273528347 9-214748364825 10-236223201304 11-257698037782 12-279172874261 13-300647710739' 2020-09-12T14:43:09.433 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:43:09.433 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.14 flush_pg_stats 2020-09-12T14:43:09.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=322122547217 2020-09-12T14:43:09.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 322122547217 2020-09-12T14:43:09.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509477 3-85899345956 4-107374182434 5-128849018912 6-150323855390 7-171798691869 8-193273528347 9-214748364825 10-236223201304 11-257698037782 12-279172874261 13-300647710739 14-322122547217' 2020-09-12T14:43:09.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:43:09.579 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.15 flush_pg_stats 2020-09-12T14:43:09.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=343597383696 2020-09-12T14:43:09.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 343597383696 2020-09-12T14:43:09.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509477 3-85899345956 4-107374182434 5-128849018912 6-150323855390 7-171798691869 8-193273528347 9-214748364825 10-236223201304 11-257698037782 12-279172874261 13-300647710739 14-322122547217 15-343597383696' 2020-09-12T14:43:09.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:43:09.723 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.16 flush_pg_stats 2020-09-12T14:43:09.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=365072220175 2020-09-12T14:43:09.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 365072220175 2020-09-12T14:43:09.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509477 3-85899345956 4-107374182434 5-128849018912 6-150323855390 7-171798691869 8-193273528347 9-214748364825 10-236223201304 11-257698037782 12-279172874261 13-300647710739 14-322122547217 15-343597383696 16-365072220175' 2020-09-12T14:43:09.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:43:09.869 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.17 flush_pg_stats 2020-09-12T14:43:10.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=386547056653 2020-09-12T14:43:10.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 386547056653 2020-09-12T14:43:10.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509477 3-85899345956 4-107374182434 5-128849018912 6-150323855390 7-171798691869 8-193273528347 9-214748364825 10-236223201304 11-257698037782 12-279172874261 13-300647710739 14-322122547217 15-343597383696 16-365072220175 17-386547056653' 2020-09-12T14:43:10.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:43:10.013 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.18 flush_pg_stats 2020-09-12T14:43:10.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=408021893130 2020-09-12T14:43:10.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 408021893130 2020-09-12T14:43:10.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509477 3-85899345956 4-107374182434 5-128849018912 6-150323855390 7-171798691869 8-193273528347 9-214748364825 10-236223201304 11-257698037782 12-279172874261 13-300647710739 14-322122547217 15-343597383696 16-365072220175 17-386547056653 18-408021893130' 2020-09-12T14:43:10.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: flush_pg_stats: for osd in $ids 2020-09-12T14:43:10.158 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: ceph tell osd.19 flush_pg_stats 2020-09-12T14:43:10.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: flush_pg_stats: seq=429496729608 2020-09-12T14:43:10.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: flush_pg_stats: test -z 429496729608 2020-09-12T14:43:10.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509477 3-85899345956 4-107374182434 5-128849018912 6-150323855390 7-171798691869 8-193273528347 9-214748364825 10-236223201304 11-257698037782 12-279172874261 13-300647710739 14-322122547217 15-343597383696 16-365072220175 17-386547056653 18-408021893130 19-429496729608' 2020-09-12T14:43:10.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:43:10.302 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 0-21474836519 2020-09-12T14:43:10.302 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:43:10.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=0 2020-09-12T14:43:10.303 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 0-21474836519 2020-09-12T14:43:10.303 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:43:10.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=21474836519 2020-09-12T14:43:10.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.0 seq 21474836519' 2020-09-12T14:43:10.304 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.0 seq 21474836519 2020-09-12T14:43:10.304 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 0 2020-09-12T14:43:10.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 21474836520 -lt 21474836519 2020-09-12T14:43:10.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:43:10.710 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 1-42949672998 2020-09-12T14:43:10.711 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:43:10.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=1 2020-09-12T14:43:10.712 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 1-42949672998 2020-09-12T14:43:10.712 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:43:10.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=42949672998 2020-09-12T14:43:10.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.1 seq 42949672998' 2020-09-12T14:43:10.713 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.1 seq 42949672998 2020-09-12T14:43:10.713 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 1 2020-09-12T14:43:11.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 42949672998 -lt 42949672998 2020-09-12T14:43:11.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:43:11.116 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 2-64424509477 2020-09-12T14:43:11.116 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:43:11.116 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=2 2020-09-12T14:43:11.117 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 2-64424509477 2020-09-12T14:43:11.117 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:43:11.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=64424509477 2020-09-12T14:43:11.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.2 seq 64424509477' 2020-09-12T14:43:11.118 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.2 seq 64424509477 2020-09-12T14:43:11.119 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 2 2020-09-12T14:43:11.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 64424509478 -lt 64424509477 2020-09-12T14:43:11.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:43:11.568 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 3-85899345956 2020-09-12T14:43:11.568 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:43:11.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=3 2020-09-12T14:43:11.569 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 3-85899345956 2020-09-12T14:43:11.569 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:43:11.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=85899345956 2020-09-12T14:43:11.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.3 seq 85899345956' 2020-09-12T14:43:11.570 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.3 seq 85899345956 2020-09-12T14:43:11.571 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 3 2020-09-12T14:43:11.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 85899345956 -lt 85899345956 2020-09-12T14:43:11.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:43:11.981 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 4-107374182434 2020-09-12T14:43:11.981 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:43:11.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=4 2020-09-12T14:43:11.982 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 4-107374182434 2020-09-12T14:43:11.982 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:43:11.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=107374182434 2020-09-12T14:43:11.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.4 seq 107374182434' 2020-09-12T14:43:11.983 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.4 seq 107374182434 2020-09-12T14:43:11.984 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 4 2020-09-12T14:43:12.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 107374182435 -lt 107374182434 2020-09-12T14:43:12.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:43:12.392 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 5-128849018912 2020-09-12T14:43:12.392 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:43:12.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=5 2020-09-12T14:43:12.393 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 5-128849018912 2020-09-12T14:43:12.393 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:43:12.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=128849018912 2020-09-12T14:43:12.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.5 seq 128849018912' 2020-09-12T14:43:12.395 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.5 seq 128849018912 2020-09-12T14:43:12.395 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 5 2020-09-12T14:43:12.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 128849018912 -lt 128849018912 2020-09-12T14:43:12.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:43:12.803 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 6-150323855390 2020-09-12T14:43:12.803 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:43:12.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=6 2020-09-12T14:43:12.804 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 6-150323855390 2020-09-12T14:43:12.804 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:43:12.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=150323855390 2020-09-12T14:43:12.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.6 seq 150323855390' 2020-09-12T14:43:12.805 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.6 seq 150323855390 2020-09-12T14:43:12.806 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 6 2020-09-12T14:43:13.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 150323855391 -lt 150323855390 2020-09-12T14:43:13.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:43:13.216 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 7-171798691869 2020-09-12T14:43:13.216 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:43:13.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=7 2020-09-12T14:43:13.217 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 7-171798691869 2020-09-12T14:43:13.217 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:43:13.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=171798691869 2020-09-12T14:43:13.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.7 seq 171798691869' 2020-09-12T14:43:13.219 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.7 seq 171798691869 2020-09-12T14:43:13.219 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 7 2020-09-12T14:43:13.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 171798691870 -lt 171798691869 2020-09-12T14:43:13.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:43:13.631 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 8-193273528347 2020-09-12T14:43:13.631 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:43:13.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=8 2020-09-12T14:43:13.632 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 8-193273528347 2020-09-12T14:43:13.632 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:43:13.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=193273528347 2020-09-12T14:43:13.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.8 seq 193273528347' 2020-09-12T14:43:13.633 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.8 seq 193273528347 2020-09-12T14:43:13.634 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 8 2020-09-12T14:43:14.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 193273528348 -lt 193273528347 2020-09-12T14:43:14.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:43:14.054 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 9-214748364825 2020-09-12T14:43:14.054 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:43:14.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=9 2020-09-12T14:43:14.055 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 9-214748364825 2020-09-12T14:43:14.055 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:43:14.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=214748364825 2020-09-12T14:43:14.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.9 seq 214748364825' 2020-09-12T14:43:14.056 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.9 seq 214748364825 2020-09-12T14:43:14.056 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 9 2020-09-12T14:43:14.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 214748364826 -lt 214748364825 2020-09-12T14:43:14.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:43:14.474 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 10-236223201304 2020-09-12T14:43:14.474 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:43:14.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=10 2020-09-12T14:43:14.475 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 10-236223201304 2020-09-12T14:43:14.475 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:43:14.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=236223201304 2020-09-12T14:43:14.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.10 seq 236223201304' 2020-09-12T14:43:14.476 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.10 seq 236223201304 2020-09-12T14:43:14.477 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 10 2020-09-12T14:43:14.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 236223201305 -lt 236223201304 2020-09-12T14:43:14.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:43:14.897 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 11-257698037782 2020-09-12T14:43:14.898 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:43:14.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=11 2020-09-12T14:43:14.898 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 11-257698037782 2020-09-12T14:43:14.899 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:43:14.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=257698037782 2020-09-12T14:43:14.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.11 seq 257698037782' 2020-09-12T14:43:14.900 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.11 seq 257698037782 2020-09-12T14:43:14.900 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 11 2020-09-12T14:43:15.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 257698037783 -lt 257698037782 2020-09-12T14:43:15.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:43:15.309 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 12-279172874261 2020-09-12T14:43:15.309 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:43:15.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=12 2020-09-12T14:43:15.310 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 12-279172874261 2020-09-12T14:43:15.311 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:43:15.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=279172874261 2020-09-12T14:43:15.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.12 seq 279172874261' 2020-09-12T14:43:15.312 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 12 2020-09-12T14:43:15.312 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.12 seq 279172874261 2020-09-12T14:43:15.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 279172874262 -lt 279172874261 2020-09-12T14:43:15.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:43:15.721 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 13-300647710739 2020-09-12T14:43:15.721 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:43:15.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=13 2020-09-12T14:43:15.722 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 13-300647710739 2020-09-12T14:43:15.722 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:43:15.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=300647710739 2020-09-12T14:43:15.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.13 seq 300647710739' 2020-09-12T14:43:15.724 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 13 2020-09-12T14:43:15.724 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.13 seq 300647710739 2020-09-12T14:43:16.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 300647710740 -lt 300647710739 2020-09-12T14:43:16.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:43:16.135 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 14-322122547217 2020-09-12T14:43:16.136 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:43:16.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=14 2020-09-12T14:43:16.136 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 14-322122547217 2020-09-12T14:43:16.137 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:43:16.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=322122547217 2020-09-12T14:43:16.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.14 seq 322122547217' 2020-09-12T14:43:16.138 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.14 seq 322122547217 2020-09-12T14:43:16.138 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 14 2020-09-12T14:43:16.552 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 322122547218 -lt 322122547217 2020-09-12T14:43:16.552 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:43:16.553 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 15-343597383696 2020-09-12T14:43:16.553 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:43:16.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=15 2020-09-12T14:43:16.554 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 15-343597383696 2020-09-12T14:43:16.554 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:43:16.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=343597383696 2020-09-12T14:43:16.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.15 seq 343597383696' 2020-09-12T14:43:16.555 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.15 seq 343597383696 2020-09-12T14:43:16.556 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 15 2020-09-12T14:43:16.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 343597383697 -lt 343597383696 2020-09-12T14:43:16.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:43:16.971 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 16-365072220175 2020-09-12T14:43:16.971 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:43:16.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=16 2020-09-12T14:43:16.972 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 16-365072220175 2020-09-12T14:43:16.972 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:43:16.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=365072220175 2020-09-12T14:43:16.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.16 seq 365072220175' 2020-09-12T14:43:16.973 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.16 seq 365072220175 2020-09-12T14:43:16.973 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 16 2020-09-12T14:43:17.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 365072220176 -lt 365072220175 2020-09-12T14:43:17.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:43:17.392 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 17-386547056653 2020-09-12T14:43:17.393 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:43:17.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=17 2020-09-12T14:43:17.393 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 17-386547056653 2020-09-12T14:43:17.394 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:43:17.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=386547056653 2020-09-12T14:43:17.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.17 seq 386547056653' 2020-09-12T14:43:17.395 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.17 seq 386547056653 2020-09-12T14:43:17.395 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 17 2020-09-12T14:43:17.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 386547056655 -lt 386547056653 2020-09-12T14:43:17.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:43:17.816 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 18-408021893130 2020-09-12T14:43:17.816 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:43:17.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=18 2020-09-12T14:43:17.817 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 18-408021893130 2020-09-12T14:43:17.817 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:43:17.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=408021893130 2020-09-12T14:43:17.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.18 seq 408021893130' 2020-09-12T14:43:17.818 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.18 seq 408021893130 2020-09-12T14:43:17.819 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 18 2020-09-12T14:43:18.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 408021893132 -lt 408021893130 2020-09-12T14:43:18.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: flush_pg_stats: for s in $seqs 2020-09-12T14:43:18.225 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: echo 19-429496729608 2020-09-12T14:43:18.225 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: cut -d - -f 1 2020-09-12T14:43:18.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: flush_pg_stats: osd=19 2020-09-12T14:43:18.226 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: echo 19-429496729608 2020-09-12T14:43:18.227 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: cut -d - -f 2 2020-09-12T14:43:18.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: flush_pg_stats: seq=429496729608 2020-09-12T14:43:18.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: flush_pg_stats: echo 'waiting osd.19 seq 429496729608' 2020-09-12T14:43:18.228 INFO:tasks.workunit.client.0.smithi027.stdout:waiting osd.19 seq 429496729608 2020-09-12T14:43:18.228 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: ceph osd last-stat-seq 19 2020-09-12T14:43:18.649 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: flush_pg_stats: test 429496729610 -lt 429496729608 2020-09-12T14:43:18.650 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: get_num_pgs 2020-09-12T14:43:18.650 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:43:18.650 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:43:19.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1587: wait_for_clean: test 8 == 0 2020-09-12T14:43:19.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1591: wait_for_clean: true 2020-09-12T14:43:19.182 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: get_num_active_clean 2020-09-12T14:43:19.183 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: get_num_active_clean: local expression 2020-09-12T14:43:19.183 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2020-09-12T14:43:19.183 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: get_num_active_clean: expression+='select(contains("stale") | not)' 2020-09-12T14:43:19.183 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: get_num_active_clean: ceph --format json pg dump pgs 2020-09-12T14:43:19.183 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2020-09-12T14:43:19.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1595: wait_for_clean: cur_active_clean=8 2020-09-12T14:43:19.592 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: get_num_pgs 2020-09-12T14:43:19.593 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: ceph --format json status 2020-09-12T14:43:19.593 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: get_num_pgs: jq .pgmap.num_pgs 2020-09-12T14:43:20.122 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: test 8 = 8 2020-09-12T14:43:20.122 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1596: wait_for_clean: break 2020-09-12T14:43:20.122 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1611: wait_for_clean: return 0 2020-09-12T14:43:20.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:214: TEST_backfill_test_multi: dd if=/dev/urandom of=td/osd-backfill-space/datafile bs=1024 count=4 2020-09-12T14:43:20.123 INFO:tasks.workunit.client.0.smithi027.stderr:4+0 records in 2020-09-12T14:43:20.123 INFO:tasks.workunit.client.0.smithi027.stderr:4+0 records out 2020-09-12T14:43:20.123 INFO:tasks.workunit.client.0.smithi027.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 7.0941e-05 s, 57.7 MB/s 2020-09-12T14:43:20.124 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: seq 1 600 2020-09-12T14:43:20.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:20.124 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:20.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:20.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj1 td/osd-backfill-space/datafile 2020-09-12T14:43:20.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:20.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj1 td/osd-backfill-space/datafile 2020-09-12T14:43:20.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:20.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj1 td/osd-backfill-space/datafile 2020-09-12T14:43:20.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:20.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj1 td/osd-backfill-space/datafile 2020-09-12T14:43:20.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:20.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj1 td/osd-backfill-space/datafile 2020-09-12T14:43:20.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:20.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj1 td/osd-backfill-space/datafile 2020-09-12T14:43:20.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:20.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj1 td/osd-backfill-space/datafile 2020-09-12T14:43:20.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:20.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj1 td/osd-backfill-space/datafile 2020-09-12T14:43:20.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:20.775 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:20.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:20.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj2 td/osd-backfill-space/datafile 2020-09-12T14:43:20.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:20.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj2 td/osd-backfill-space/datafile 2020-09-12T14:43:20.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:20.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj2 td/osd-backfill-space/datafile 2020-09-12T14:43:21.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:21.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj2 td/osd-backfill-space/datafile 2020-09-12T14:43:21.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:21.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj2 td/osd-backfill-space/datafile 2020-09-12T14:43:21.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:21.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj2 td/osd-backfill-space/datafile 2020-09-12T14:43:21.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:21.236 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj2 td/osd-backfill-space/datafile 2020-09-12T14:43:21.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:21.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj2 td/osd-backfill-space/datafile 2020-09-12T14:43:21.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:21.387 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:21.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:21.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj3 td/osd-backfill-space/datafile 2020-09-12T14:43:21.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:21.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj3 td/osd-backfill-space/datafile 2020-09-12T14:43:21.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:21.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj3 td/osd-backfill-space/datafile 2020-09-12T14:43:21.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:21.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj3 td/osd-backfill-space/datafile 2020-09-12T14:43:21.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:21.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj3 td/osd-backfill-space/datafile 2020-09-12T14:43:21.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:21.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj3 td/osd-backfill-space/datafile 2020-09-12T14:43:21.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:21.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj3 td/osd-backfill-space/datafile 2020-09-12T14:43:22.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:22.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj3 td/osd-backfill-space/datafile 2020-09-12T14:43:22.091 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:22.091 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:22.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:22.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj4 td/osd-backfill-space/datafile 2020-09-12T14:43:22.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:22.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj4 td/osd-backfill-space/datafile 2020-09-12T14:43:22.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:22.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj4 td/osd-backfill-space/datafile 2020-09-12T14:43:22.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:22.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj4 td/osd-backfill-space/datafile 2020-09-12T14:43:22.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:22.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj4 td/osd-backfill-space/datafile 2020-09-12T14:43:22.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:22.477 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj4 td/osd-backfill-space/datafile 2020-09-12T14:43:22.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:22.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj4 td/osd-backfill-space/datafile 2020-09-12T14:43:22.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:22.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj4 td/osd-backfill-space/datafile 2020-09-12T14:43:22.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:22.711 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:22.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:22.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj5 td/osd-backfill-space/datafile 2020-09-12T14:43:22.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:22.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj5 td/osd-backfill-space/datafile 2020-09-12T14:43:22.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:22.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj5 td/osd-backfill-space/datafile 2020-09-12T14:43:22.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:22.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj5 td/osd-backfill-space/datafile 2020-09-12T14:43:23.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:23.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj5 td/osd-backfill-space/datafile 2020-09-12T14:43:23.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:23.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj5 td/osd-backfill-space/datafile 2020-09-12T14:43:23.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:23.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj5 td/osd-backfill-space/datafile 2020-09-12T14:43:23.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:23.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj5 td/osd-backfill-space/datafile 2020-09-12T14:43:23.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:23.323 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:23.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:23.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj6 td/osd-backfill-space/datafile 2020-09-12T14:43:23.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:23.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj6 td/osd-backfill-space/datafile 2020-09-12T14:43:23.498 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:23.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj6 td/osd-backfill-space/datafile 2020-09-12T14:43:23.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:23.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj6 td/osd-backfill-space/datafile 2020-09-12T14:43:23.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:23.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj6 td/osd-backfill-space/datafile 2020-09-12T14:43:23.767 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:23.767 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj6 td/osd-backfill-space/datafile 2020-09-12T14:43:23.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:23.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj6 td/osd-backfill-space/datafile 2020-09-12T14:43:23.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:23.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj6 td/osd-backfill-space/datafile 2020-09-12T14:43:23.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:23.993 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:23.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:23.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj7 td/osd-backfill-space/datafile 2020-09-12T14:43:24.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:24.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj7 td/osd-backfill-space/datafile 2020-09-12T14:43:24.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:24.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj7 td/osd-backfill-space/datafile 2020-09-12T14:43:24.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:24.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj7 td/osd-backfill-space/datafile 2020-09-12T14:43:24.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:24.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj7 td/osd-backfill-space/datafile 2020-09-12T14:43:24.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:24.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj7 td/osd-backfill-space/datafile 2020-09-12T14:43:24.470 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:24.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj7 td/osd-backfill-space/datafile 2020-09-12T14:43:24.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:24.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj7 td/osd-backfill-space/datafile 2020-09-12T14:43:24.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:24.630 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:24.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:24.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj8 td/osd-backfill-space/datafile 2020-09-12T14:43:24.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:24.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj8 td/osd-backfill-space/datafile 2020-09-12T14:43:24.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:24.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj8 td/osd-backfill-space/datafile 2020-09-12T14:43:24.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:24.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj8 td/osd-backfill-space/datafile 2020-09-12T14:43:24.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:24.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj8 td/osd-backfill-space/datafile 2020-09-12T14:43:25.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:25.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj8 td/osd-backfill-space/datafile 2020-09-12T14:43:25.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:25.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj8 td/osd-backfill-space/datafile 2020-09-12T14:43:25.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:25.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj8 td/osd-backfill-space/datafile 2020-09-12T14:43:25.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:25.275 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:25.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:25.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj9 td/osd-backfill-space/datafile 2020-09-12T14:43:25.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:25.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj9 td/osd-backfill-space/datafile 2020-09-12T14:43:25.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:25.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj9 td/osd-backfill-space/datafile 2020-09-12T14:43:25.551 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:25.552 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj9 td/osd-backfill-space/datafile 2020-09-12T14:43:25.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:25.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj9 td/osd-backfill-space/datafile 2020-09-12T14:43:25.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:25.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj9 td/osd-backfill-space/datafile 2020-09-12T14:43:25.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:25.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj9 td/osd-backfill-space/datafile 2020-09-12T14:43:25.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:25.887 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj9 td/osd-backfill-space/datafile 2020-09-12T14:43:25.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:25.962 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:25.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:25.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj10 td/osd-backfill-space/datafile 2020-09-12T14:43:26.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:26.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj10 td/osd-backfill-space/datafile 2020-09-12T14:43:26.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:26.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj10 td/osd-backfill-space/datafile 2020-09-12T14:43:26.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:26.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj10 td/osd-backfill-space/datafile 2020-09-12T14:43:26.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:26.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj10 td/osd-backfill-space/datafile 2020-09-12T14:43:26.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:26.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj10 td/osd-backfill-space/datafile 2020-09-12T14:43:26.431 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:26.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj10 td/osd-backfill-space/datafile 2020-09-12T14:43:26.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:26.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj10 td/osd-backfill-space/datafile 2020-09-12T14:43:26.590 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:26.590 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:26.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:26.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj11 td/osd-backfill-space/datafile 2020-09-12T14:43:26.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:26.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj11 td/osd-backfill-space/datafile 2020-09-12T14:43:26.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:26.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj11 td/osd-backfill-space/datafile 2020-09-12T14:43:26.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:26.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj11 td/osd-backfill-space/datafile 2020-09-12T14:43:26.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:26.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj11 td/osd-backfill-space/datafile 2020-09-12T14:43:27.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:27.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj11 td/osd-backfill-space/datafile 2020-09-12T14:43:27.110 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:27.110 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj11 td/osd-backfill-space/datafile 2020-09-12T14:43:27.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:27.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj11 td/osd-backfill-space/datafile 2020-09-12T14:43:27.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:27.269 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:27.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:27.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj12 td/osd-backfill-space/datafile 2020-09-12T14:43:27.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:27.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj12 td/osd-backfill-space/datafile 2020-09-12T14:43:27.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:27.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj12 td/osd-backfill-space/datafile 2020-09-12T14:43:27.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:27.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj12 td/osd-backfill-space/datafile 2020-09-12T14:43:27.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:27.597 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj12 td/osd-backfill-space/datafile 2020-09-12T14:43:27.672 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:27.672 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj12 td/osd-backfill-space/datafile 2020-09-12T14:43:27.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:27.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj12 td/osd-backfill-space/datafile 2020-09-12T14:43:27.864 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:27.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj12 td/osd-backfill-space/datafile 2020-09-12T14:43:27.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:27.940 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:27.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:27.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj13 td/osd-backfill-space/datafile 2020-09-12T14:43:28.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:28.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj13 td/osd-backfill-space/datafile 2020-09-12T14:43:28.090 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:28.091 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj13 td/osd-backfill-space/datafile 2020-09-12T14:43:28.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:28.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj13 td/osd-backfill-space/datafile 2020-09-12T14:43:28.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:28.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj13 td/osd-backfill-space/datafile 2020-09-12T14:43:28.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:28.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj13 td/osd-backfill-space/datafile 2020-09-12T14:43:28.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:28.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj13 td/osd-backfill-space/datafile 2020-09-12T14:43:28.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:28.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj13 td/osd-backfill-space/datafile 2020-09-12T14:43:28.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:28.560 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:28.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:28.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj14 td/osd-backfill-space/datafile 2020-09-12T14:43:28.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:28.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj14 td/osd-backfill-space/datafile 2020-09-12T14:43:28.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:28.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj14 td/osd-backfill-space/datafile 2020-09-12T14:43:28.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:28.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj14 td/osd-backfill-space/datafile 2020-09-12T14:43:28.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:28.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj14 td/osd-backfill-space/datafile 2020-09-12T14:43:28.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:28.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj14 td/osd-backfill-space/datafile 2020-09-12T14:43:29.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:29.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj14 td/osd-backfill-space/datafile 2020-09-12T14:43:29.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:29.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj14 td/osd-backfill-space/datafile 2020-09-12T14:43:29.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:29.180 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:29.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:29.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj15 td/osd-backfill-space/datafile 2020-09-12T14:43:29.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:29.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj15 td/osd-backfill-space/datafile 2020-09-12T14:43:29.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:29.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj15 td/osd-backfill-space/datafile 2020-09-12T14:43:29.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:29.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj15 td/osd-backfill-space/datafile 2020-09-12T14:43:29.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:29.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj15 td/osd-backfill-space/datafile 2020-09-12T14:43:29.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:29.583 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj15 td/osd-backfill-space/datafile 2020-09-12T14:43:29.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:29.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj15 td/osd-backfill-space/datafile 2020-09-12T14:43:29.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:29.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj15 td/osd-backfill-space/datafile 2020-09-12T14:43:29.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:29.826 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:29.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:29.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj16 td/osd-backfill-space/datafile 2020-09-12T14:43:29.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:29.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj16 td/osd-backfill-space/datafile 2020-09-12T14:43:29.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:29.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj16 td/osd-backfill-space/datafile 2020-09-12T14:43:30.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:30.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj16 td/osd-backfill-space/datafile 2020-09-12T14:43:30.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:30.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj16 td/osd-backfill-space/datafile 2020-09-12T14:43:30.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:30.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj16 td/osd-backfill-space/datafile 2020-09-12T14:43:30.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:30.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj16 td/osd-backfill-space/datafile 2020-09-12T14:43:30.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:30.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj16 td/osd-backfill-space/datafile 2020-09-12T14:43:30.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:30.588 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:30.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:30.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj17 td/osd-backfill-space/datafile 2020-09-12T14:43:30.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:30.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj17 td/osd-backfill-space/datafile 2020-09-12T14:43:30.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:30.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj17 td/osd-backfill-space/datafile 2020-09-12T14:43:30.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:30.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj17 td/osd-backfill-space/datafile 2020-09-12T14:43:30.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:30.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj17 td/osd-backfill-space/datafile 2020-09-12T14:43:31.014 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:31.014 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj17 td/osd-backfill-space/datafile 2020-09-12T14:43:31.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:31.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj17 td/osd-backfill-space/datafile 2020-09-12T14:43:31.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:31.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj17 td/osd-backfill-space/datafile 2020-09-12T14:43:31.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:31.232 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:31.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:31.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj18 td/osd-backfill-space/datafile 2020-09-12T14:43:31.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:31.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj18 td/osd-backfill-space/datafile 2020-09-12T14:43:31.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:31.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj18 td/osd-backfill-space/datafile 2020-09-12T14:43:31.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:31.517 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj18 td/osd-backfill-space/datafile 2020-09-12T14:43:31.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:31.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj18 td/osd-backfill-space/datafile 2020-09-12T14:43:31.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:31.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj18 td/osd-backfill-space/datafile 2020-09-12T14:43:31.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:31.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj18 td/osd-backfill-space/datafile 2020-09-12T14:43:31.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:31.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj18 td/osd-backfill-space/datafile 2020-09-12T14:43:31.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:31.987 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:31.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:31.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj19 td/osd-backfill-space/datafile 2020-09-12T14:43:32.062 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:32.062 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj19 td/osd-backfill-space/datafile 2020-09-12T14:43:32.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:32.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj19 td/osd-backfill-space/datafile 2020-09-12T14:43:32.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:32.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj19 td/osd-backfill-space/datafile 2020-09-12T14:43:32.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:32.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj19 td/osd-backfill-space/datafile 2020-09-12T14:43:32.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:32.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj19 td/osd-backfill-space/datafile 2020-09-12T14:43:32.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:32.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj19 td/osd-backfill-space/datafile 2020-09-12T14:43:32.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:32.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj19 td/osd-backfill-space/datafile 2020-09-12T14:43:32.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:32.632 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:32.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:32.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj20 td/osd-backfill-space/datafile 2020-09-12T14:43:32.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:32.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj20 td/osd-backfill-space/datafile 2020-09-12T14:43:32.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:32.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj20 td/osd-backfill-space/datafile 2020-09-12T14:43:32.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:32.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj20 td/osd-backfill-space/datafile 2020-09-12T14:43:32.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:32.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj20 td/osd-backfill-space/datafile 2020-09-12T14:43:33.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:33.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj20 td/osd-backfill-space/datafile 2020-09-12T14:43:33.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:33.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj20 td/osd-backfill-space/datafile 2020-09-12T14:43:33.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:33.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj20 td/osd-backfill-space/datafile 2020-09-12T14:43:33.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:33.260 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:33.261 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:33.261 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj21 td/osd-backfill-space/datafile 2020-09-12T14:43:33.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:33.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj21 td/osd-backfill-space/datafile 2020-09-12T14:43:33.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:33.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj21 td/osd-backfill-space/datafile 2020-09-12T14:43:33.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:33.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj21 td/osd-backfill-space/datafile 2020-09-12T14:43:33.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:33.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj21 td/osd-backfill-space/datafile 2020-09-12T14:43:33.662 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:33.662 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj21 td/osd-backfill-space/datafile 2020-09-12T14:43:33.762 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:33.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj21 td/osd-backfill-space/datafile 2020-09-12T14:43:33.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:33.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj21 td/osd-backfill-space/datafile 2020-09-12T14:43:33.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:33.905 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:33.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:33.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj22 td/osd-backfill-space/datafile 2020-09-12T14:43:33.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:33.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj22 td/osd-backfill-space/datafile 2020-09-12T14:43:34.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:34.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj22 td/osd-backfill-space/datafile 2020-09-12T14:43:34.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:34.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj22 td/osd-backfill-space/datafile 2020-09-12T14:43:34.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:34.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj22 td/osd-backfill-space/datafile 2020-09-12T14:43:34.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:34.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj22 td/osd-backfill-space/datafile 2020-09-12T14:43:34.366 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:34.366 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj22 td/osd-backfill-space/datafile 2020-09-12T14:43:34.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:34.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj22 td/osd-backfill-space/datafile 2020-09-12T14:43:34.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:34.526 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:34.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:34.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj23 td/osd-backfill-space/datafile 2020-09-12T14:43:34.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:34.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj23 td/osd-backfill-space/datafile 2020-09-12T14:43:34.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:34.677 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj23 td/osd-backfill-space/datafile 2020-09-12T14:43:34.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:34.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj23 td/osd-backfill-space/datafile 2020-09-12T14:43:34.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:34.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj23 td/osd-backfill-space/datafile 2020-09-12T14:43:34.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:34.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj23 td/osd-backfill-space/datafile 2020-09-12T14:43:34.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:34.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj23 td/osd-backfill-space/datafile 2020-09-12T14:43:35.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:35.053 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj23 td/osd-backfill-space/datafile 2020-09-12T14:43:35.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:35.146 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:35.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:35.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj24 td/osd-backfill-space/datafile 2020-09-12T14:43:35.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:35.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj24 td/osd-backfill-space/datafile 2020-09-12T14:43:35.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:35.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj24 td/osd-backfill-space/datafile 2020-09-12T14:43:35.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:35.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj24 td/osd-backfill-space/datafile 2020-09-12T14:43:35.482 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:35.482 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj24 td/osd-backfill-space/datafile 2020-09-12T14:43:35.557 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:35.558 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj24 td/osd-backfill-space/datafile 2020-09-12T14:43:35.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:35.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj24 td/osd-backfill-space/datafile 2020-09-12T14:43:35.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:35.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj24 td/osd-backfill-space/datafile 2020-09-12T14:43:35.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:35.943 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:35.943 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:35.943 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj25 td/osd-backfill-space/datafile 2020-09-12T14:43:36.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:36.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj25 td/osd-backfill-space/datafile 2020-09-12T14:43:36.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:36.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj25 td/osd-backfill-space/datafile 2020-09-12T14:43:36.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:36.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj25 td/osd-backfill-space/datafile 2020-09-12T14:43:36.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:36.261 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj25 td/osd-backfill-space/datafile 2020-09-12T14:43:36.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:36.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj25 td/osd-backfill-space/datafile 2020-09-12T14:43:36.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:36.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj25 td/osd-backfill-space/datafile 2020-09-12T14:43:36.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:36.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj25 td/osd-backfill-space/datafile 2020-09-12T14:43:36.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:36.563 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:36.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:36.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj26 td/osd-backfill-space/datafile 2020-09-12T14:43:36.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:36.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj26 td/osd-backfill-space/datafile 2020-09-12T14:43:36.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:36.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj26 td/osd-backfill-space/datafile 2020-09-12T14:43:36.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:36.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj26 td/osd-backfill-space/datafile 2020-09-12T14:43:36.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:36.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj26 td/osd-backfill-space/datafile 2020-09-12T14:43:37.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:37.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj26 td/osd-backfill-space/datafile 2020-09-12T14:43:37.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:37.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj26 td/osd-backfill-space/datafile 2020-09-12T14:43:37.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:37.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj26 td/osd-backfill-space/datafile 2020-09-12T14:43:37.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:37.233 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:37.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:37.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj27 td/osd-backfill-space/datafile 2020-09-12T14:43:37.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:37.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj27 td/osd-backfill-space/datafile 2020-09-12T14:43:37.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:37.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj27 td/osd-backfill-space/datafile 2020-09-12T14:43:37.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:37.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj27 td/osd-backfill-space/datafile 2020-09-12T14:43:37.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:37.752 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj27 td/osd-backfill-space/datafile 2020-09-12T14:43:38.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:38.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj27 td/osd-backfill-space/datafile 2020-09-12T14:43:38.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:38.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj27 td/osd-backfill-space/datafile 2020-09-12T14:43:38.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:38.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj27 td/osd-backfill-space/datafile 2020-09-12T14:43:38.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:38.289 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:38.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:38.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj28 td/osd-backfill-space/datafile 2020-09-12T14:43:38.363 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:38.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj28 td/osd-backfill-space/datafile 2020-09-12T14:43:38.447 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:38.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj28 td/osd-backfill-space/datafile 2020-09-12T14:43:38.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:38.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj28 td/osd-backfill-space/datafile 2020-09-12T14:43:38.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:38.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj28 td/osd-backfill-space/datafile 2020-09-12T14:43:38.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:38.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj28 td/osd-backfill-space/datafile 2020-09-12T14:43:38.766 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:38.766 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj28 td/osd-backfill-space/datafile 2020-09-12T14:43:38.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:38.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj28 td/osd-backfill-space/datafile 2020-09-12T14:43:38.908 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:38.909 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:38.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:38.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj29 td/osd-backfill-space/datafile 2020-09-12T14:43:38.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:38.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj29 td/osd-backfill-space/datafile 2020-09-12T14:43:39.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:39.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj29 td/osd-backfill-space/datafile 2020-09-12T14:43:39.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:39.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj29 td/osd-backfill-space/datafile 2020-09-12T14:43:39.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:39.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj29 td/osd-backfill-space/datafile 2020-09-12T14:43:39.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:39.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj29 td/osd-backfill-space/datafile 2020-09-12T14:43:39.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:39.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj29 td/osd-backfill-space/datafile 2020-09-12T14:43:39.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:39.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj29 td/osd-backfill-space/datafile 2020-09-12T14:43:39.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:39.554 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:39.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:39.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj30 td/osd-backfill-space/datafile 2020-09-12T14:43:39.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:39.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj30 td/osd-backfill-space/datafile 2020-09-12T14:43:39.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:39.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj30 td/osd-backfill-space/datafile 2020-09-12T14:43:39.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:39.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj30 td/osd-backfill-space/datafile 2020-09-12T14:43:39.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:39.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj30 td/osd-backfill-space/datafile 2020-09-12T14:43:39.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:39.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj30 td/osd-backfill-space/datafile 2020-09-12T14:43:40.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:40.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj30 td/osd-backfill-space/datafile 2020-09-12T14:43:40.106 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:40.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj30 td/osd-backfill-space/datafile 2020-09-12T14:43:40.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:40.191 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:40.191 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:40.191 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj31 td/osd-backfill-space/datafile 2020-09-12T14:43:40.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:40.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj31 td/osd-backfill-space/datafile 2020-09-12T14:43:40.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:40.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj31 td/osd-backfill-space/datafile 2020-09-12T14:43:40.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:40.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj31 td/osd-backfill-space/datafile 2020-09-12T14:43:40.508 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:40.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj31 td/osd-backfill-space/datafile 2020-09-12T14:43:40.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:40.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj31 td/osd-backfill-space/datafile 2020-09-12T14:43:40.668 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:40.668 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj31 td/osd-backfill-space/datafile 2020-09-12T14:43:40.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:40.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj31 td/osd-backfill-space/datafile 2020-09-12T14:43:40.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:40.869 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:40.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:40.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj32 td/osd-backfill-space/datafile 2020-09-12T14:43:40.944 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:40.944 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj32 td/osd-backfill-space/datafile 2020-09-12T14:43:41.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:41.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj32 td/osd-backfill-space/datafile 2020-09-12T14:43:41.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:41.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj32 td/osd-backfill-space/datafile 2020-09-12T14:43:41.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:41.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj32 td/osd-backfill-space/datafile 2020-09-12T14:43:41.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:41.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj32 td/osd-backfill-space/datafile 2020-09-12T14:43:41.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:41.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj32 td/osd-backfill-space/datafile 2020-09-12T14:43:41.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:41.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj32 td/osd-backfill-space/datafile 2020-09-12T14:43:41.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:41.514 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:41.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:41.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj33 td/osd-backfill-space/datafile 2020-09-12T14:43:41.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:41.590 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj33 td/osd-backfill-space/datafile 2020-09-12T14:43:41.665 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:41.665 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj33 td/osd-backfill-space/datafile 2020-09-12T14:43:41.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:41.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj33 td/osd-backfill-space/datafile 2020-09-12T14:43:41.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:41.892 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj33 td/osd-backfill-space/datafile 2020-09-12T14:43:41.966 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:41.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj33 td/osd-backfill-space/datafile 2020-09-12T14:43:42.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:42.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj33 td/osd-backfill-space/datafile 2020-09-12T14:43:42.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:42.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj33 td/osd-backfill-space/datafile 2020-09-12T14:43:42.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:42.202 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:42.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:42.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj34 td/osd-backfill-space/datafile 2020-09-12T14:43:42.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:42.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj34 td/osd-backfill-space/datafile 2020-09-12T14:43:42.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:42.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj34 td/osd-backfill-space/datafile 2020-09-12T14:43:42.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:42.436 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj34 td/osd-backfill-space/datafile 2020-09-12T14:43:42.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:42.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj34 td/osd-backfill-space/datafile 2020-09-12T14:43:42.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:42.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj34 td/osd-backfill-space/datafile 2020-09-12T14:43:42.662 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:42.662 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj34 td/osd-backfill-space/datafile 2020-09-12T14:43:42.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:42.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj34 td/osd-backfill-space/datafile 2020-09-12T14:43:42.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:42.813 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:42.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:42.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj35 td/osd-backfill-space/datafile 2020-09-12T14:43:42.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:42.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj35 td/osd-backfill-space/datafile 2020-09-12T14:43:42.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:42.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj35 td/osd-backfill-space/datafile 2020-09-12T14:43:43.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:43.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj35 td/osd-backfill-space/datafile 2020-09-12T14:43:43.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:43.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj35 td/osd-backfill-space/datafile 2020-09-12T14:43:43.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:43.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj35 td/osd-backfill-space/datafile 2020-09-12T14:43:43.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:43.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj35 td/osd-backfill-space/datafile 2020-09-12T14:43:43.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:43.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj35 td/osd-backfill-space/datafile 2020-09-12T14:43:43.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:43.442 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:43.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:43.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj36 td/osd-backfill-space/datafile 2020-09-12T14:43:43.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:43.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj36 td/osd-backfill-space/datafile 2020-09-12T14:43:43.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:43.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj36 td/osd-backfill-space/datafile 2020-09-12T14:43:43.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:43.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj36 td/osd-backfill-space/datafile 2020-09-12T14:43:43.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:43.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj36 td/osd-backfill-space/datafile 2020-09-12T14:43:43.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:43.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj36 td/osd-backfill-space/datafile 2020-09-12T14:43:43.953 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:43.953 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj36 td/osd-backfill-space/datafile 2020-09-12T14:43:44.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:44.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj36 td/osd-backfill-space/datafile 2020-09-12T14:43:44.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:44.104 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:44.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:44.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj37 td/osd-backfill-space/datafile 2020-09-12T14:43:44.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:44.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj37 td/osd-backfill-space/datafile 2020-09-12T14:43:44.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:44.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj37 td/osd-backfill-space/datafile 2020-09-12T14:43:44.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:44.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj37 td/osd-backfill-space/datafile 2020-09-12T14:43:44.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:44.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj37 td/osd-backfill-space/datafile 2020-09-12T14:43:44.480 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:44.481 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj37 td/osd-backfill-space/datafile 2020-09-12T14:43:44.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:44.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj37 td/osd-backfill-space/datafile 2020-09-12T14:43:44.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:44.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj37 td/osd-backfill-space/datafile 2020-09-12T14:43:44.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:44.716 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:44.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:44.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj38 td/osd-backfill-space/datafile 2020-09-12T14:43:44.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:44.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj38 td/osd-backfill-space/datafile 2020-09-12T14:43:44.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:45.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj38 td/osd-backfill-space/datafile 2020-09-12T14:43:45.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:45.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj38 td/osd-backfill-space/datafile 2020-09-12T14:43:45.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:45.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj38 td/osd-backfill-space/datafile 2020-09-12T14:43:45.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:45.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj38 td/osd-backfill-space/datafile 2020-09-12T14:43:45.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:45.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj38 td/osd-backfill-space/datafile 2020-09-12T14:43:45.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:45.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj38 td/osd-backfill-space/datafile 2020-09-12T14:43:45.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:45.336 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:45.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:45.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj39 td/osd-backfill-space/datafile 2020-09-12T14:43:45.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:45.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj39 td/osd-backfill-space/datafile 2020-09-12T14:43:45.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:45.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj39 td/osd-backfill-space/datafile 2020-09-12T14:43:45.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:45.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj39 td/osd-backfill-space/datafile 2020-09-12T14:43:45.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:45.671 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj39 td/osd-backfill-space/datafile 2020-09-12T14:43:45.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:45.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj39 td/osd-backfill-space/datafile 2020-09-12T14:43:45.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:45.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj39 td/osd-backfill-space/datafile 2020-09-12T14:43:45.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:45.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj39 td/osd-backfill-space/datafile 2020-09-12T14:43:46.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:46.022 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:46.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:46.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj40 td/osd-backfill-space/datafile 2020-09-12T14:43:46.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:46.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj40 td/osd-backfill-space/datafile 2020-09-12T14:43:46.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:46.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj40 td/osd-backfill-space/datafile 2020-09-12T14:43:46.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:46.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj40 td/osd-backfill-space/datafile 2020-09-12T14:43:46.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:46.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj40 td/osd-backfill-space/datafile 2020-09-12T14:43:46.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:46.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj40 td/osd-backfill-space/datafile 2020-09-12T14:43:46.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:46.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj40 td/osd-backfill-space/datafile 2020-09-12T14:43:46.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:46.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj40 td/osd-backfill-space/datafile 2020-09-12T14:43:46.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:46.651 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:46.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:46.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj41 td/osd-backfill-space/datafile 2020-09-12T14:43:46.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:46.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj41 td/osd-backfill-space/datafile 2020-09-12T14:43:46.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:46.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj41 td/osd-backfill-space/datafile 2020-09-12T14:43:46.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:46.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj41 td/osd-backfill-space/datafile 2020-09-12T14:43:47.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:47.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj41 td/osd-backfill-space/datafile 2020-09-12T14:43:47.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:47.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj41 td/osd-backfill-space/datafile 2020-09-12T14:43:47.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:47.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj41 td/osd-backfill-space/datafile 2020-09-12T14:43:47.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:47.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj41 td/osd-backfill-space/datafile 2020-09-12T14:43:47.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:47.313 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:47.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:47.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj42 td/osd-backfill-space/datafile 2020-09-12T14:43:47.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:47.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj42 td/osd-backfill-space/datafile 2020-09-12T14:43:47.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:47.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj42 td/osd-backfill-space/datafile 2020-09-12T14:43:47.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:47.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj42 td/osd-backfill-space/datafile 2020-09-12T14:43:47.664 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:47.665 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj42 td/osd-backfill-space/datafile 2020-09-12T14:43:47.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:47.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj42 td/osd-backfill-space/datafile 2020-09-12T14:43:47.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:47.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj42 td/osd-backfill-space/datafile 2020-09-12T14:43:47.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:47.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj42 td/osd-backfill-space/datafile 2020-09-12T14:43:48.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:48.000 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:48.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:48.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj43 td/osd-backfill-space/datafile 2020-09-12T14:43:48.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:48.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj43 td/osd-backfill-space/datafile 2020-09-12T14:43:48.151 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:48.151 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj43 td/osd-backfill-space/datafile 2020-09-12T14:43:48.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:48.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj43 td/osd-backfill-space/datafile 2020-09-12T14:43:48.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:48.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj43 td/osd-backfill-space/datafile 2020-09-12T14:43:48.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:48.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj43 td/osd-backfill-space/datafile 2020-09-12T14:43:48.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:48.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj43 td/osd-backfill-space/datafile 2020-09-12T14:43:48.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:48.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj43 td/osd-backfill-space/datafile 2020-09-12T14:43:48.612 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:48.612 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:48.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:48.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj44 td/osd-backfill-space/datafile 2020-09-12T14:43:48.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:48.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj44 td/osd-backfill-space/datafile 2020-09-12T14:43:48.762 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:48.762 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj44 td/osd-backfill-space/datafile 2020-09-12T14:43:48.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:48.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj44 td/osd-backfill-space/datafile 2020-09-12T14:43:48.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:48.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj44 td/osd-backfill-space/datafile 2020-09-12T14:43:48.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:48.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj44 td/osd-backfill-space/datafile 2020-09-12T14:43:49.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:49.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj44 td/osd-backfill-space/datafile 2020-09-12T14:43:49.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:49.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj44 td/osd-backfill-space/datafile 2020-09-12T14:43:49.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:49.206 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:49.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:49.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj45 td/osd-backfill-space/datafile 2020-09-12T14:43:49.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:49.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj45 td/osd-backfill-space/datafile 2020-09-12T14:43:49.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:49.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj45 td/osd-backfill-space/datafile 2020-09-12T14:43:49.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:49.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj45 td/osd-backfill-space/datafile 2020-09-12T14:43:49.558 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:49.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj45 td/osd-backfill-space/datafile 2020-09-12T14:43:49.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:49.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj45 td/osd-backfill-space/datafile 2020-09-12T14:43:49.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:49.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj45 td/osd-backfill-space/datafile 2020-09-12T14:43:49.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:49.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj45 td/osd-backfill-space/datafile 2020-09-12T14:43:49.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:49.894 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:49.894 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:49.894 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj46 td/osd-backfill-space/datafile 2020-09-12T14:43:49.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:49.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj46 td/osd-backfill-space/datafile 2020-09-12T14:43:50.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:50.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj46 td/osd-backfill-space/datafile 2020-09-12T14:43:50.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:50.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj46 td/osd-backfill-space/datafile 2020-09-12T14:43:50.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:50.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj46 td/osd-backfill-space/datafile 2020-09-12T14:43:50.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:50.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj46 td/osd-backfill-space/datafile 2020-09-12T14:43:50.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:50.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj46 td/osd-backfill-space/datafile 2020-09-12T14:43:50.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:50.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj46 td/osd-backfill-space/datafile 2020-09-12T14:43:50.496 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:50.497 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:50.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:50.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj47 td/osd-backfill-space/datafile 2020-09-12T14:43:50.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:50.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj47 td/osd-backfill-space/datafile 2020-09-12T14:43:50.664 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:50.665 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj47 td/osd-backfill-space/datafile 2020-09-12T14:43:50.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:50.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj47 td/osd-backfill-space/datafile 2020-09-12T14:43:50.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:50.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj47 td/osd-backfill-space/datafile 2020-09-12T14:43:50.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:50.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj47 td/osd-backfill-space/datafile 2020-09-12T14:43:50.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:50.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj47 td/osd-backfill-space/datafile 2020-09-12T14:43:51.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:51.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj47 td/osd-backfill-space/datafile 2020-09-12T14:43:51.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:51.259 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:51.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:51.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj48 td/osd-backfill-space/datafile 2020-09-12T14:43:51.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:51.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj48 td/osd-backfill-space/datafile 2020-09-12T14:43:51.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:51.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj48 td/osd-backfill-space/datafile 2020-09-12T14:43:51.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:51.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj48 td/osd-backfill-space/datafile 2020-09-12T14:43:51.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:51.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj48 td/osd-backfill-space/datafile 2020-09-12T14:43:51.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:51.653 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj48 td/osd-backfill-space/datafile 2020-09-12T14:43:51.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:51.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj48 td/osd-backfill-space/datafile 2020-09-12T14:43:51.887 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:51.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj48 td/osd-backfill-space/datafile 2020-09-12T14:43:51.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:51.963 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:51.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:51.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj49 td/osd-backfill-space/datafile 2020-09-12T14:43:52.038 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:52.038 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj49 td/osd-backfill-space/datafile 2020-09-12T14:43:52.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:52.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj49 td/osd-backfill-space/datafile 2020-09-12T14:43:52.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:52.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj49 td/osd-backfill-space/datafile 2020-09-12T14:43:52.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:52.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj49 td/osd-backfill-space/datafile 2020-09-12T14:43:52.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:52.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj49 td/osd-backfill-space/datafile 2020-09-12T14:43:52.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:52.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj49 td/osd-backfill-space/datafile 2020-09-12T14:43:52.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:52.508 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj49 td/osd-backfill-space/datafile 2020-09-12T14:43:52.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:52.583 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:52.583 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:52.584 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj50 td/osd-backfill-space/datafile 2020-09-12T14:43:52.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:52.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj50 td/osd-backfill-space/datafile 2020-09-12T14:43:52.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:52.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj50 td/osd-backfill-space/datafile 2020-09-12T14:43:52.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:52.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj50 td/osd-backfill-space/datafile 2020-09-12T14:43:52.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:52.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj50 td/osd-backfill-space/datafile 2020-09-12T14:43:52.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:52.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj50 td/osd-backfill-space/datafile 2020-09-12T14:43:53.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:53.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj50 td/osd-backfill-space/datafile 2020-09-12T14:43:53.119 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:53.119 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj50 td/osd-backfill-space/datafile 2020-09-12T14:43:53.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:53.194 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:53.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:53.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj51 td/osd-backfill-space/datafile 2020-09-12T14:43:53.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:53.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj51 td/osd-backfill-space/datafile 2020-09-12T14:43:53.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:53.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj51 td/osd-backfill-space/datafile 2020-09-12T14:43:53.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:53.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj51 td/osd-backfill-space/datafile 2020-09-12T14:43:53.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:53.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj51 td/osd-backfill-space/datafile 2020-09-12T14:43:53.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:53.597 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj51 td/osd-backfill-space/datafile 2020-09-12T14:43:53.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:53.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj51 td/osd-backfill-space/datafile 2020-09-12T14:43:53.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:53.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj51 td/osd-backfill-space/datafile 2020-09-12T14:43:53.864 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:53.865 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:53.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:53.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj52 td/osd-backfill-space/datafile 2020-09-12T14:43:53.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:53.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj52 td/osd-backfill-space/datafile 2020-09-12T14:43:54.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:54.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj52 td/osd-backfill-space/datafile 2020-09-12T14:43:54.091 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:54.091 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj52 td/osd-backfill-space/datafile 2020-09-12T14:43:54.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:54.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj52 td/osd-backfill-space/datafile 2020-09-12T14:43:54.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:54.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj52 td/osd-backfill-space/datafile 2020-09-12T14:43:54.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:54.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj52 td/osd-backfill-space/datafile 2020-09-12T14:43:54.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:54.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj52 td/osd-backfill-space/datafile 2020-09-12T14:43:54.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:54.485 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:54.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:54.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj53 td/osd-backfill-space/datafile 2020-09-12T14:43:54.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:54.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj53 td/osd-backfill-space/datafile 2020-09-12T14:43:54.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:54.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj53 td/osd-backfill-space/datafile 2020-09-12T14:43:54.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:54.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj53 td/osd-backfill-space/datafile 2020-09-12T14:43:54.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:54.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj53 td/osd-backfill-space/datafile 2020-09-12T14:43:54.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:54.871 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj53 td/osd-backfill-space/datafile 2020-09-12T14:43:54.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:54.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj53 td/osd-backfill-space/datafile 2020-09-12T14:43:55.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:55.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj53 td/osd-backfill-space/datafile 2020-09-12T14:43:55.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:55.097 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:55.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:55.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj54 td/osd-backfill-space/datafile 2020-09-12T14:43:55.172 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:55.172 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj54 td/osd-backfill-space/datafile 2020-09-12T14:43:55.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:55.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj54 td/osd-backfill-space/datafile 2020-09-12T14:43:55.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:55.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj54 td/osd-backfill-space/datafile 2020-09-12T14:43:55.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:55.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj54 td/osd-backfill-space/datafile 2020-09-12T14:43:55.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:55.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj54 td/osd-backfill-space/datafile 2020-09-12T14:43:55.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:55.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj54 td/osd-backfill-space/datafile 2020-09-12T14:43:55.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:55.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj54 td/osd-backfill-space/datafile 2020-09-12T14:43:55.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:55.784 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:55.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:55.785 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj55 td/osd-backfill-space/datafile 2020-09-12T14:43:55.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:55.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj55 td/osd-backfill-space/datafile 2020-09-12T14:43:55.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:55.943 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj55 td/osd-backfill-space/datafile 2020-09-12T14:43:56.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:56.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj55 td/osd-backfill-space/datafile 2020-09-12T14:43:56.093 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:56.094 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj55 td/osd-backfill-space/datafile 2020-09-12T14:43:56.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:56.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj55 td/osd-backfill-space/datafile 2020-09-12T14:43:56.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:56.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj55 td/osd-backfill-space/datafile 2020-09-12T14:43:56.378 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:56.378 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj55 td/osd-backfill-space/datafile 2020-09-12T14:43:56.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:56.454 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:56.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:56.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj56 td/osd-backfill-space/datafile 2020-09-12T14:43:56.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:56.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj56 td/osd-backfill-space/datafile 2020-09-12T14:43:56.604 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:56.604 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj56 td/osd-backfill-space/datafile 2020-09-12T14:43:56.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:56.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj56 td/osd-backfill-space/datafile 2020-09-12T14:43:56.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:56.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj56 td/osd-backfill-space/datafile 2020-09-12T14:43:56.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:56.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj56 td/osd-backfill-space/datafile 2020-09-12T14:43:56.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:56.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj56 td/osd-backfill-space/datafile 2020-09-12T14:43:57.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:57.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj56 td/osd-backfill-space/datafile 2020-09-12T14:43:57.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:57.099 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:57.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:57.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj57 td/osd-backfill-space/datafile 2020-09-12T14:43:57.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:57.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj57 td/osd-backfill-space/datafile 2020-09-12T14:43:57.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:57.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj57 td/osd-backfill-space/datafile 2020-09-12T14:43:57.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:57.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj57 td/osd-backfill-space/datafile 2020-09-12T14:43:57.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:57.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj57 td/osd-backfill-space/datafile 2020-09-12T14:43:57.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:57.510 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj57 td/osd-backfill-space/datafile 2020-09-12T14:43:57.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:57.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj57 td/osd-backfill-space/datafile 2020-09-12T14:43:57.661 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:57.661 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj57 td/osd-backfill-space/datafile 2020-09-12T14:43:57.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:57.778 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:57.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:57.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj58 td/osd-backfill-space/datafile 2020-09-12T14:43:57.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:57.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj58 td/osd-backfill-space/datafile 2020-09-12T14:43:57.928 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:57.929 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj58 td/osd-backfill-space/datafile 2020-09-12T14:43:58.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:58.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj58 td/osd-backfill-space/datafile 2020-09-12T14:43:58.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:58.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj58 td/osd-backfill-space/datafile 2020-09-12T14:43:58.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:58.172 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj58 td/osd-backfill-space/datafile 2020-09-12T14:43:58.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:58.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj58 td/osd-backfill-space/datafile 2020-09-12T14:43:58.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:58.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj58 td/osd-backfill-space/datafile 2020-09-12T14:43:58.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:58.415 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:58.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:58.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj59 td/osd-backfill-space/datafile 2020-09-12T14:43:58.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:58.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj59 td/osd-backfill-space/datafile 2020-09-12T14:43:58.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:58.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj59 td/osd-backfill-space/datafile 2020-09-12T14:43:58.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:58.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj59 td/osd-backfill-space/datafile 2020-09-12T14:43:58.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:58.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj59 td/osd-backfill-space/datafile 2020-09-12T14:43:58.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:58.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj59 td/osd-backfill-space/datafile 2020-09-12T14:43:58.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:58.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj59 td/osd-backfill-space/datafile 2020-09-12T14:43:58.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:58.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj59 td/osd-backfill-space/datafile 2020-09-12T14:43:59.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:59.026 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:59.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:59.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj60 td/osd-backfill-space/datafile 2020-09-12T14:43:59.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:59.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj60 td/osd-backfill-space/datafile 2020-09-12T14:43:59.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:59.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj60 td/osd-backfill-space/datafile 2020-09-12T14:43:59.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:59.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj60 td/osd-backfill-space/datafile 2020-09-12T14:43:59.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:59.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj60 td/osd-backfill-space/datafile 2020-09-12T14:43:59.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:59.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj60 td/osd-backfill-space/datafile 2020-09-12T14:43:59.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:59.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj60 td/osd-backfill-space/datafile 2020-09-12T14:43:59.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:59.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj60 td/osd-backfill-space/datafile 2020-09-12T14:43:59.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:43:59.688 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:43:59.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:59.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj61 td/osd-backfill-space/datafile 2020-09-12T14:43:59.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:59.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj61 td/osd-backfill-space/datafile 2020-09-12T14:43:59.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:59.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj61 td/osd-backfill-space/datafile 2020-09-12T14:43:59.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:43:59.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj61 td/osd-backfill-space/datafile 2020-09-12T14:44:00.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:00.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj61 td/osd-backfill-space/datafile 2020-09-12T14:44:00.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:00.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj61 td/osd-backfill-space/datafile 2020-09-12T14:44:00.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:00.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj61 td/osd-backfill-space/datafile 2020-09-12T14:44:00.291 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:00.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj61 td/osd-backfill-space/datafile 2020-09-12T14:44:00.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:00.367 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:00.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:00.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj62 td/osd-backfill-space/datafile 2020-09-12T14:44:00.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:00.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj62 td/osd-backfill-space/datafile 2020-09-12T14:44:00.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:00.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj62 td/osd-backfill-space/datafile 2020-09-12T14:44:00.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:00.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj62 td/osd-backfill-space/datafile 2020-09-12T14:44:00.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:00.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj62 td/osd-backfill-space/datafile 2020-09-12T14:44:00.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:00.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj62 td/osd-backfill-space/datafile 2020-09-12T14:44:00.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:00.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj62 td/osd-backfill-space/datafile 2020-09-12T14:44:00.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:00.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj62 td/osd-backfill-space/datafile 2020-09-12T14:44:01.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:01.004 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:01.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:01.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj63 td/osd-backfill-space/datafile 2020-09-12T14:44:01.079 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:01.079 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj63 td/osd-backfill-space/datafile 2020-09-12T14:44:01.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:01.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj63 td/osd-backfill-space/datafile 2020-09-12T14:44:01.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:01.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj63 td/osd-backfill-space/datafile 2020-09-12T14:44:01.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:01.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj63 td/osd-backfill-space/datafile 2020-09-12T14:44:01.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:01.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj63 td/osd-backfill-space/datafile 2020-09-12T14:44:01.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:01.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj63 td/osd-backfill-space/datafile 2020-09-12T14:44:02.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:02.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj63 td/osd-backfill-space/datafile 2020-09-12T14:44:02.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:02.142 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:02.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:02.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj64 td/osd-backfill-space/datafile 2020-09-12T14:44:02.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:02.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj64 td/osd-backfill-space/datafile 2020-09-12T14:44:02.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:02.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj64 td/osd-backfill-space/datafile 2020-09-12T14:44:02.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:02.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj64 td/osd-backfill-space/datafile 2020-09-12T14:44:02.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:02.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj64 td/osd-backfill-space/datafile 2020-09-12T14:44:02.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:02.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj64 td/osd-backfill-space/datafile 2020-09-12T14:44:02.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:02.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj64 td/osd-backfill-space/datafile 2020-09-12T14:44:02.678 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:02.678 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj64 td/osd-backfill-space/datafile 2020-09-12T14:44:02.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:02.753 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:02.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:02.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj65 td/osd-backfill-space/datafile 2020-09-12T14:44:02.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:02.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj65 td/osd-backfill-space/datafile 2020-09-12T14:44:02.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:02.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj65 td/osd-backfill-space/datafile 2020-09-12T14:44:02.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:02.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj65 td/osd-backfill-space/datafile 2020-09-12T14:44:03.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:03.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj65 td/osd-backfill-space/datafile 2020-09-12T14:44:03.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:03.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj65 td/osd-backfill-space/datafile 2020-09-12T14:44:03.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:03.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj65 td/osd-backfill-space/datafile 2020-09-12T14:44:03.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:03.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj65 td/osd-backfill-space/datafile 2020-09-12T14:44:03.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:03.390 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:03.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:03.391 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj66 td/osd-backfill-space/datafile 2020-09-12T14:44:03.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:03.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj66 td/osd-backfill-space/datafile 2020-09-12T14:44:03.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:03.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj66 td/osd-backfill-space/datafile 2020-09-12T14:44:03.666 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:03.667 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj66 td/osd-backfill-space/datafile 2020-09-12T14:44:03.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:03.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj66 td/osd-backfill-space/datafile 2020-09-12T14:44:03.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:03.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj66 td/osd-backfill-space/datafile 2020-09-12T14:44:03.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:03.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj66 td/osd-backfill-space/datafile 2020-09-12T14:44:04.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:04.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj66 td/osd-backfill-space/datafile 2020-09-12T14:44:04.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:04.078 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:04.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:04.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj67 td/osd-backfill-space/datafile 2020-09-12T14:44:04.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:04.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj67 td/osd-backfill-space/datafile 2020-09-12T14:44:04.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:04.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj67 td/osd-backfill-space/datafile 2020-09-12T14:44:04.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:04.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj67 td/osd-backfill-space/datafile 2020-09-12T14:44:04.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:04.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj67 td/osd-backfill-space/datafile 2020-09-12T14:44:04.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:04.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj67 td/osd-backfill-space/datafile 2020-09-12T14:44:04.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:04.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj67 td/osd-backfill-space/datafile 2020-09-12T14:44:04.621 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:04.621 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj67 td/osd-backfill-space/datafile 2020-09-12T14:44:04.688 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:04.689 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:04.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:04.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj68 td/osd-backfill-space/datafile 2020-09-12T14:44:04.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:04.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj68 td/osd-backfill-space/datafile 2020-09-12T14:44:04.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:04.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj68 td/osd-backfill-space/datafile 2020-09-12T14:44:04.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:04.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj68 td/osd-backfill-space/datafile 2020-09-12T14:44:05.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:05.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj68 td/osd-backfill-space/datafile 2020-09-12T14:44:05.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:05.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj68 td/osd-backfill-space/datafile 2020-09-12T14:44:05.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:05.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj68 td/osd-backfill-space/datafile 2020-09-12T14:44:05.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:05.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj68 td/osd-backfill-space/datafile 2020-09-12T14:44:05.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:05.350 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:05.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:05.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj69 td/osd-backfill-space/datafile 2020-09-12T14:44:05.434 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:05.434 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj69 td/osd-backfill-space/datafile 2020-09-12T14:44:05.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:05.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj69 td/osd-backfill-space/datafile 2020-09-12T14:44:05.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:05.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj69 td/osd-backfill-space/datafile 2020-09-12T14:44:05.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:05.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj69 td/osd-backfill-space/datafile 2020-09-12T14:44:05.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:05.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj69 td/osd-backfill-space/datafile 2020-09-12T14:44:05.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:05.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj69 td/osd-backfill-space/datafile 2020-09-12T14:44:05.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:05.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj69 td/osd-backfill-space/datafile 2020-09-12T14:44:06.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:06.029 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:06.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:06.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj70 td/osd-backfill-space/datafile 2020-09-12T14:44:06.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:06.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj70 td/osd-backfill-space/datafile 2020-09-12T14:44:06.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:06.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj70 td/osd-backfill-space/datafile 2020-09-12T14:44:06.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:06.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj70 td/osd-backfill-space/datafile 2020-09-12T14:44:06.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:06.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj70 td/osd-backfill-space/datafile 2020-09-12T14:44:06.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:06.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj70 td/osd-backfill-space/datafile 2020-09-12T14:44:06.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:06.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj70 td/osd-backfill-space/datafile 2020-09-12T14:44:06.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:06.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj70 td/osd-backfill-space/datafile 2020-09-12T14:44:06.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:06.641 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:06.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:06.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj71 td/osd-backfill-space/datafile 2020-09-12T14:44:06.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:06.733 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj71 td/osd-backfill-space/datafile 2020-09-12T14:44:06.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:06.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj71 td/osd-backfill-space/datafile 2020-09-12T14:44:06.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:06.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj71 td/osd-backfill-space/datafile 2020-09-12T14:44:07.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:07.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj71 td/osd-backfill-space/datafile 2020-09-12T14:44:07.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:07.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj71 td/osd-backfill-space/datafile 2020-09-12T14:44:07.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:07.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj71 td/osd-backfill-space/datafile 2020-09-12T14:44:07.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:07.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj71 td/osd-backfill-space/datafile 2020-09-12T14:44:07.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:07.403 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:07.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:07.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj72 td/osd-backfill-space/datafile 2020-09-12T14:44:07.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:07.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj72 td/osd-backfill-space/datafile 2020-09-12T14:44:07.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:07.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj72 td/osd-backfill-space/datafile 2020-09-12T14:44:07.662 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:07.663 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj72 td/osd-backfill-space/datafile 2020-09-12T14:44:07.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:07.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj72 td/osd-backfill-space/datafile 2020-09-12T14:44:07.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:07.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj72 td/osd-backfill-space/datafile 2020-09-12T14:44:07.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:07.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj72 td/osd-backfill-space/datafile 2020-09-12T14:44:08.014 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:08.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj72 td/osd-backfill-space/datafile 2020-09-12T14:44:08.090 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:08.091 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:08.091 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:08.091 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj73 td/osd-backfill-space/datafile 2020-09-12T14:44:08.165 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:08.165 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj73 td/osd-backfill-space/datafile 2020-09-12T14:44:08.240 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:08.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj73 td/osd-backfill-space/datafile 2020-09-12T14:44:08.316 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:08.316 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj73 td/osd-backfill-space/datafile 2020-09-12T14:44:08.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:08.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj73 td/osd-backfill-space/datafile 2020-09-12T14:44:08.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:08.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj73 td/osd-backfill-space/datafile 2020-09-12T14:44:08.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:08.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj73 td/osd-backfill-space/datafile 2020-09-12T14:44:08.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:08.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj73 td/osd-backfill-space/datafile 2020-09-12T14:44:08.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:08.702 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:08.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:08.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj74 td/osd-backfill-space/datafile 2020-09-12T14:44:08.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:08.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj74 td/osd-backfill-space/datafile 2020-09-12T14:44:08.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:08.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj74 td/osd-backfill-space/datafile 2020-09-12T14:44:08.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:08.928 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj74 td/osd-backfill-space/datafile 2020-09-12T14:44:09.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:09.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj74 td/osd-backfill-space/datafile 2020-09-12T14:44:09.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:09.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj74 td/osd-backfill-space/datafile 2020-09-12T14:44:09.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:09.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj74 td/osd-backfill-space/datafile 2020-09-12T14:44:09.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:09.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj74 td/osd-backfill-space/datafile 2020-09-12T14:44:09.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:09.322 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:09.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:09.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj75 td/osd-backfill-space/datafile 2020-09-12T14:44:09.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:09.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj75 td/osd-backfill-space/datafile 2020-09-12T14:44:09.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:09.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj75 td/osd-backfill-space/datafile 2020-09-12T14:44:09.590 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:09.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj75 td/osd-backfill-space/datafile 2020-09-12T14:44:09.666 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:09.666 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj75 td/osd-backfill-space/datafile 2020-09-12T14:44:09.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:09.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj75 td/osd-backfill-space/datafile 2020-09-12T14:44:09.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:09.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj75 td/osd-backfill-space/datafile 2020-09-12T14:44:09.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:09.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj75 td/osd-backfill-space/datafile 2020-09-12T14:44:10.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:10.001 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:10.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:10.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj76 td/osd-backfill-space/datafile 2020-09-12T14:44:10.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:10.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj76 td/osd-backfill-space/datafile 2020-09-12T14:44:10.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:10.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj76 td/osd-backfill-space/datafile 2020-09-12T14:44:10.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:10.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj76 td/osd-backfill-space/datafile 2020-09-12T14:44:10.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:10.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj76 td/osd-backfill-space/datafile 2020-09-12T14:44:10.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:10.378 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj76 td/osd-backfill-space/datafile 2020-09-12T14:44:10.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:10.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj76 td/osd-backfill-space/datafile 2020-09-12T14:44:10.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:10.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj76 td/osd-backfill-space/datafile 2020-09-12T14:44:10.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:10.596 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:10.597 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:10.597 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj77 td/osd-backfill-space/datafile 2020-09-12T14:44:10.671 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:10.671 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj77 td/osd-backfill-space/datafile 2020-09-12T14:44:10.746 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:10.747 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj77 td/osd-backfill-space/datafile 2020-09-12T14:44:10.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:10.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj77 td/osd-backfill-space/datafile 2020-09-12T14:44:10.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:10.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj77 td/osd-backfill-space/datafile 2020-09-12T14:44:10.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:10.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj77 td/osd-backfill-space/datafile 2020-09-12T14:44:11.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:11.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj77 td/osd-backfill-space/datafile 2020-09-12T14:44:11.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:11.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj77 td/osd-backfill-space/datafile 2020-09-12T14:44:11.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:11.211 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:11.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:11.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj78 td/osd-backfill-space/datafile 2020-09-12T14:44:11.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:11.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj78 td/osd-backfill-space/datafile 2020-09-12T14:44:11.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:11.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj78 td/osd-backfill-space/datafile 2020-09-12T14:44:11.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:11.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj78 td/osd-backfill-space/datafile 2020-09-12T14:44:11.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:11.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj78 td/osd-backfill-space/datafile 2020-09-12T14:44:11.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:11.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj78 td/osd-backfill-space/datafile 2020-09-12T14:44:11.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:11.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj78 td/osd-backfill-space/datafile 2020-09-12T14:44:11.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:11.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj78 td/osd-backfill-space/datafile 2020-09-12T14:44:11.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:11.936 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:11.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:11.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj79 td/osd-backfill-space/datafile 2020-09-12T14:44:12.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:12.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj79 td/osd-backfill-space/datafile 2020-09-12T14:44:12.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:12.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj79 td/osd-backfill-space/datafile 2020-09-12T14:44:12.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:12.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj79 td/osd-backfill-space/datafile 2020-09-12T14:44:12.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:12.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj79 td/osd-backfill-space/datafile 2020-09-12T14:44:12.355 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:12.355 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj79 td/osd-backfill-space/datafile 2020-09-12T14:44:12.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:12.431 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj79 td/osd-backfill-space/datafile 2020-09-12T14:44:12.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:12.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj79 td/osd-backfill-space/datafile 2020-09-12T14:44:12.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:12.582 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:12.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:12.583 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj80 td/osd-backfill-space/datafile 2020-09-12T14:44:12.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:12.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj80 td/osd-backfill-space/datafile 2020-09-12T14:44:12.740 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:12.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj80 td/osd-backfill-space/datafile 2020-09-12T14:44:12.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:12.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj80 td/osd-backfill-space/datafile 2020-09-12T14:44:12.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:12.892 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj80 td/osd-backfill-space/datafile 2020-09-12T14:44:12.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:12.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj80 td/osd-backfill-space/datafile 2020-09-12T14:44:13.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:13.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj80 td/osd-backfill-space/datafile 2020-09-12T14:44:13.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:13.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj80 td/osd-backfill-space/datafile 2020-09-12T14:44:13.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:13.193 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:13.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:13.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj81 td/osd-backfill-space/datafile 2020-09-12T14:44:13.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:13.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj81 td/osd-backfill-space/datafile 2020-09-12T14:44:13.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:13.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj81 td/osd-backfill-space/datafile 2020-09-12T14:44:13.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:13.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj81 td/osd-backfill-space/datafile 2020-09-12T14:44:13.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:13.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj81 td/osd-backfill-space/datafile 2020-09-12T14:44:13.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:13.604 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj81 td/osd-backfill-space/datafile 2020-09-12T14:44:13.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:13.671 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj81 td/osd-backfill-space/datafile 2020-09-12T14:44:13.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:13.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj81 td/osd-backfill-space/datafile 2020-09-12T14:44:13.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:13.872 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:13.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:13.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj82 td/osd-backfill-space/datafile 2020-09-12T14:44:13.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:13.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj82 td/osd-backfill-space/datafile 2020-09-12T14:44:14.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:14.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj82 td/osd-backfill-space/datafile 2020-09-12T14:44:14.106 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:14.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj82 td/osd-backfill-space/datafile 2020-09-12T14:44:14.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:14.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj82 td/osd-backfill-space/datafile 2020-09-12T14:44:14.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:14.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj82 td/osd-backfill-space/datafile 2020-09-12T14:44:14.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:14.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj82 td/osd-backfill-space/datafile 2020-09-12T14:44:14.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:14.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj82 td/osd-backfill-space/datafile 2020-09-12T14:44:14.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:14.492 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:14.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:14.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj83 td/osd-backfill-space/datafile 2020-09-12T14:44:14.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:14.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj83 td/osd-backfill-space/datafile 2020-09-12T14:44:14.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:14.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj83 td/osd-backfill-space/datafile 2020-09-12T14:44:14.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:14.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj83 td/osd-backfill-space/datafile 2020-09-12T14:44:14.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:14.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj83 td/osd-backfill-space/datafile 2020-09-12T14:44:14.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:14.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj83 td/osd-backfill-space/datafile 2020-09-12T14:44:14.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:14.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj83 td/osd-backfill-space/datafile 2020-09-12T14:44:15.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:15.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj83 td/osd-backfill-space/datafile 2020-09-12T14:44:15.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:15.120 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:15.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:15.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj84 td/osd-backfill-space/datafile 2020-09-12T14:44:15.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:15.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj84 td/osd-backfill-space/datafile 2020-09-12T14:44:15.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:15.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj84 td/osd-backfill-space/datafile 2020-09-12T14:44:15.363 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:15.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj84 td/osd-backfill-space/datafile 2020-09-12T14:44:15.438 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:15.439 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj84 td/osd-backfill-space/datafile 2020-09-12T14:44:15.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:15.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj84 td/osd-backfill-space/datafile 2020-09-12T14:44:15.606 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:15.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj84 td/osd-backfill-space/datafile 2020-09-12T14:44:15.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:15.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj84 td/osd-backfill-space/datafile 2020-09-12T14:44:15.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:15.792 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:15.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:15.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj85 td/osd-backfill-space/datafile 2020-09-12T14:44:15.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:15.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj85 td/osd-backfill-space/datafile 2020-09-12T14:44:15.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:15.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj85 td/osd-backfill-space/datafile 2020-09-12T14:44:16.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:16.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj85 td/osd-backfill-space/datafile 2020-09-12T14:44:16.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:16.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj85 td/osd-backfill-space/datafile 2020-09-12T14:44:16.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:16.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj85 td/osd-backfill-space/datafile 2020-09-12T14:44:16.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:16.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj85 td/osd-backfill-space/datafile 2020-09-12T14:44:16.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:16.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj85 td/osd-backfill-space/datafile 2020-09-12T14:44:16.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:16.394 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:16.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:16.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj86 td/osd-backfill-space/datafile 2020-09-12T14:44:16.477 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:16.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj86 td/osd-backfill-space/datafile 2020-09-12T14:44:16.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:16.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj86 td/osd-backfill-space/datafile 2020-09-12T14:44:16.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:16.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj86 td/osd-backfill-space/datafile 2020-09-12T14:44:16.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:16.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj86 td/osd-backfill-space/datafile 2020-09-12T14:44:16.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:16.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj86 td/osd-backfill-space/datafile 2020-09-12T14:44:16.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:16.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj86 td/osd-backfill-space/datafile 2020-09-12T14:44:16.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:16.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj86 td/osd-backfill-space/datafile 2020-09-12T14:44:17.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:17.040 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:17.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:17.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj87 td/osd-backfill-space/datafile 2020-09-12T14:44:17.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:17.115 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj87 td/osd-backfill-space/datafile 2020-09-12T14:44:17.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:17.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj87 td/osd-backfill-space/datafile 2020-09-12T14:44:17.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:17.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj87 td/osd-backfill-space/datafile 2020-09-12T14:44:17.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:17.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj87 td/osd-backfill-space/datafile 2020-09-12T14:44:17.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:17.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj87 td/osd-backfill-space/datafile 2020-09-12T14:44:17.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:17.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj87 td/osd-backfill-space/datafile 2020-09-12T14:44:17.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:17.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj87 td/osd-backfill-space/datafile 2020-09-12T14:44:17.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:17.677 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:17.677 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:17.677 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj88 td/osd-backfill-space/datafile 2020-09-12T14:44:17.785 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:17.785 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj88 td/osd-backfill-space/datafile 2020-09-12T14:44:17.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:17.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj88 td/osd-backfill-space/datafile 2020-09-12T14:44:17.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:17.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj88 td/osd-backfill-space/datafile 2020-09-12T14:44:18.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:18.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj88 td/osd-backfill-space/datafile 2020-09-12T14:44:18.086 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:18.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj88 td/osd-backfill-space/datafile 2020-09-12T14:44:18.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:18.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj88 td/osd-backfill-space/datafile 2020-09-12T14:44:18.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:18.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj88 td/osd-backfill-space/datafile 2020-09-12T14:44:18.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:18.330 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:18.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:18.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj89 td/osd-backfill-space/datafile 2020-09-12T14:44:18.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:18.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj89 td/osd-backfill-space/datafile 2020-09-12T14:44:18.480 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:18.480 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj89 td/osd-backfill-space/datafile 2020-09-12T14:44:18.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:18.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj89 td/osd-backfill-space/datafile 2020-09-12T14:44:18.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:18.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj89 td/osd-backfill-space/datafile 2020-09-12T14:44:18.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:18.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj89 td/osd-backfill-space/datafile 2020-09-12T14:44:18.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:18.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj89 td/osd-backfill-space/datafile 2020-09-12T14:44:18.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:18.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj89 td/osd-backfill-space/datafile 2020-09-12T14:44:18.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:18.942 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:18.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:18.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj90 td/osd-backfill-space/datafile 2020-09-12T14:44:19.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:19.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj90 td/osd-backfill-space/datafile 2020-09-12T14:44:19.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:19.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj90 td/osd-backfill-space/datafile 2020-09-12T14:44:19.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:19.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj90 td/osd-backfill-space/datafile 2020-09-12T14:44:19.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:19.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj90 td/osd-backfill-space/datafile 2020-09-12T14:44:19.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:19.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj90 td/osd-backfill-space/datafile 2020-09-12T14:44:19.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:19.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj90 td/osd-backfill-space/datafile 2020-09-12T14:44:19.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:19.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj90 td/osd-backfill-space/datafile 2020-09-12T14:44:19.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:19.604 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:19.604 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:19.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj91 td/osd-backfill-space/datafile 2020-09-12T14:44:19.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:19.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj91 td/osd-backfill-space/datafile 2020-09-12T14:44:19.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:19.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj91 td/osd-backfill-space/datafile 2020-09-12T14:44:19.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:19.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj91 td/osd-backfill-space/datafile 2020-09-12T14:44:19.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:19.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj91 td/osd-backfill-space/datafile 2020-09-12T14:44:20.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:20.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj91 td/osd-backfill-space/datafile 2020-09-12T14:44:20.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:20.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj91 td/osd-backfill-space/datafile 2020-09-12T14:44:20.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:20.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj91 td/osd-backfill-space/datafile 2020-09-12T14:44:20.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:20.232 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:20.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:20.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj92 td/osd-backfill-space/datafile 2020-09-12T14:44:20.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:20.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj92 td/osd-backfill-space/datafile 2020-09-12T14:44:20.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:20.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj92 td/osd-backfill-space/datafile 2020-09-12T14:44:20.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:20.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj92 td/osd-backfill-space/datafile 2020-09-12T14:44:20.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:20.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj92 td/osd-backfill-space/datafile 2020-09-12T14:44:20.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:20.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj92 td/osd-backfill-space/datafile 2020-09-12T14:44:20.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:20.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj92 td/osd-backfill-space/datafile 2020-09-12T14:44:20.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:20.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj92 td/osd-backfill-space/datafile 2020-09-12T14:44:20.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:20.844 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:20.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:20.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj93 td/osd-backfill-space/datafile 2020-09-12T14:44:20.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:20.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj93 td/osd-backfill-space/datafile 2020-09-12T14:44:20.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:20.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj93 td/osd-backfill-space/datafile 2020-09-12T14:44:21.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:21.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj93 td/osd-backfill-space/datafile 2020-09-12T14:44:21.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:21.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj93 td/osd-backfill-space/datafile 2020-09-12T14:44:21.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:21.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj93 td/osd-backfill-space/datafile 2020-09-12T14:44:21.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:21.298 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj93 td/osd-backfill-space/datafile 2020-09-12T14:44:21.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:21.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj93 td/osd-backfill-space/datafile 2020-09-12T14:44:21.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:21.490 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:21.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:21.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj94 td/osd-backfill-space/datafile 2020-09-12T14:44:21.565 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:21.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj94 td/osd-backfill-space/datafile 2020-09-12T14:44:21.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:21.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj94 td/osd-backfill-space/datafile 2020-09-12T14:44:21.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:21.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj94 td/osd-backfill-space/datafile 2020-09-12T14:44:21.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:21.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj94 td/osd-backfill-space/datafile 2020-09-12T14:44:21.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:21.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj94 td/osd-backfill-space/datafile 2020-09-12T14:44:22.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:22.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj94 td/osd-backfill-space/datafile 2020-09-12T14:44:22.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:22.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj94 td/osd-backfill-space/datafile 2020-09-12T14:44:22.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:22.153 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:22.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:22.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj95 td/osd-backfill-space/datafile 2020-09-12T14:44:22.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:22.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj95 td/osd-backfill-space/datafile 2020-09-12T14:44:22.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:22.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj95 td/osd-backfill-space/datafile 2020-09-12T14:44:22.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:22.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj95 td/osd-backfill-space/datafile 2020-09-12T14:44:22.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:22.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj95 td/osd-backfill-space/datafile 2020-09-12T14:44:22.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:22.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj95 td/osd-backfill-space/datafile 2020-09-12T14:44:22.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:22.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj95 td/osd-backfill-space/datafile 2020-09-12T14:44:22.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:22.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj95 td/osd-backfill-space/datafile 2020-09-12T14:44:22.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:22.857 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:22.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:22.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj96 td/osd-backfill-space/datafile 2020-09-12T14:44:22.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:22.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj96 td/osd-backfill-space/datafile 2020-09-12T14:44:23.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:23.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj96 td/osd-backfill-space/datafile 2020-09-12T14:44:23.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:23.083 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj96 td/osd-backfill-space/datafile 2020-09-12T14:44:23.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:23.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj96 td/osd-backfill-space/datafile 2020-09-12T14:44:23.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:23.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj96 td/osd-backfill-space/datafile 2020-09-12T14:44:23.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:23.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj96 td/osd-backfill-space/datafile 2020-09-12T14:44:23.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:23.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj96 td/osd-backfill-space/datafile 2020-09-12T14:44:23.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:23.519 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:23.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:23.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj97 td/osd-backfill-space/datafile 2020-09-12T14:44:23.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:23.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj97 td/osd-backfill-space/datafile 2020-09-12T14:44:23.677 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:23.677 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj97 td/osd-backfill-space/datafile 2020-09-12T14:44:23.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:23.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj97 td/osd-backfill-space/datafile 2020-09-12T14:44:23.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:23.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj97 td/osd-backfill-space/datafile 2020-09-12T14:44:23.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:23.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj97 td/osd-backfill-space/datafile 2020-09-12T14:44:24.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:24.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj97 td/osd-backfill-space/datafile 2020-09-12T14:44:24.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:24.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj97 td/osd-backfill-space/datafile 2020-09-12T14:44:24.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:24.180 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:24.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:24.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj98 td/osd-backfill-space/datafile 2020-09-12T14:44:24.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:24.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj98 td/osd-backfill-space/datafile 2020-09-12T14:44:24.347 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:24.347 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj98 td/osd-backfill-space/datafile 2020-09-12T14:44:24.422 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:24.423 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj98 td/osd-backfill-space/datafile 2020-09-12T14:44:24.498 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:24.498 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj98 td/osd-backfill-space/datafile 2020-09-12T14:44:24.573 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:24.573 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj98 td/osd-backfill-space/datafile 2020-09-12T14:44:24.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:24.649 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj98 td/osd-backfill-space/datafile 2020-09-12T14:44:24.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:24.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj98 td/osd-backfill-space/datafile 2020-09-12T14:44:24.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:24.808 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:24.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:24.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj99 td/osd-backfill-space/datafile 2020-09-12T14:44:24.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:24.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj99 td/osd-backfill-space/datafile 2020-09-12T14:44:24.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:24.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj99 td/osd-backfill-space/datafile 2020-09-12T14:44:25.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:25.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj99 td/osd-backfill-space/datafile 2020-09-12T14:44:25.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:25.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj99 td/osd-backfill-space/datafile 2020-09-12T14:44:25.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:25.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj99 td/osd-backfill-space/datafile 2020-09-12T14:44:25.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:25.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj99 td/osd-backfill-space/datafile 2020-09-12T14:44:25.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:25.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj99 td/osd-backfill-space/datafile 2020-09-12T14:44:25.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:25.411 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:25.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:25.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj100 td/osd-backfill-space/datafile 2020-09-12T14:44:25.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:25.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj100 td/osd-backfill-space/datafile 2020-09-12T14:44:25.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:25.604 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj100 td/osd-backfill-space/datafile 2020-09-12T14:44:25.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:25.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj100 td/osd-backfill-space/datafile 2020-09-12T14:44:25.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:25.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj100 td/osd-backfill-space/datafile 2020-09-12T14:44:25.871 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:25.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj100 td/osd-backfill-space/datafile 2020-09-12T14:44:25.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:25.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj100 td/osd-backfill-space/datafile 2020-09-12T14:44:26.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:26.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj100 td/osd-backfill-space/datafile 2020-09-12T14:44:26.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:26.098 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:26.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:26.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj101 td/osd-backfill-space/datafile 2020-09-12T14:44:26.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:26.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj101 td/osd-backfill-space/datafile 2020-09-12T14:44:26.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:26.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj101 td/osd-backfill-space/datafile 2020-09-12T14:44:26.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:26.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj101 td/osd-backfill-space/datafile 2020-09-12T14:44:26.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:26.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj101 td/osd-backfill-space/datafile 2020-09-12T14:44:26.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:26.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj101 td/osd-backfill-space/datafile 2020-09-12T14:44:26.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:26.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj101 td/osd-backfill-space/datafile 2020-09-12T14:44:26.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:26.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj101 td/osd-backfill-space/datafile 2020-09-12T14:44:26.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:26.727 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:26.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:26.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj102 td/osd-backfill-space/datafile 2020-09-12T14:44:26.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:26.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj102 td/osd-backfill-space/datafile 2020-09-12T14:44:26.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:26.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj102 td/osd-backfill-space/datafile 2020-09-12T14:44:26.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:26.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj102 td/osd-backfill-space/datafile 2020-09-12T14:44:27.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:27.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj102 td/osd-backfill-space/datafile 2020-09-12T14:44:27.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:27.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj102 td/osd-backfill-space/datafile 2020-09-12T14:44:27.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:27.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj102 td/osd-backfill-space/datafile 2020-09-12T14:44:27.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:27.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj102 td/osd-backfill-space/datafile 2020-09-12T14:44:27.433 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:27.434 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:27.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:27.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj103 td/osd-backfill-space/datafile 2020-09-12T14:44:27.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:27.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj103 td/osd-backfill-space/datafile 2020-09-12T14:44:27.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:27.606 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj103 td/osd-backfill-space/datafile 2020-09-12T14:44:27.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:27.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj103 td/osd-backfill-space/datafile 2020-09-12T14:44:27.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:27.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj103 td/osd-backfill-space/datafile 2020-09-12T14:44:27.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:27.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj103 td/osd-backfill-space/datafile 2020-09-12T14:44:27.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:27.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj103 td/osd-backfill-space/datafile 2020-09-12T14:44:28.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:28.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj103 td/osd-backfill-space/datafile 2020-09-12T14:44:28.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:28.092 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:28.093 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:28.093 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj104 td/osd-backfill-space/datafile 2020-09-12T14:44:28.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:28.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj104 td/osd-backfill-space/datafile 2020-09-12T14:44:28.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:28.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj104 td/osd-backfill-space/datafile 2020-09-12T14:44:28.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:28.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj104 td/osd-backfill-space/datafile 2020-09-12T14:44:28.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:28.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj104 td/osd-backfill-space/datafile 2020-09-12T14:44:28.485 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:28.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj104 td/osd-backfill-space/datafile 2020-09-12T14:44:28.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:28.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj104 td/osd-backfill-space/datafile 2020-09-12T14:44:28.644 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:28.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj104 td/osd-backfill-space/datafile 2020-09-12T14:44:28.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:28.720 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:28.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:28.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj105 td/osd-backfill-space/datafile 2020-09-12T14:44:28.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:28.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj105 td/osd-backfill-space/datafile 2020-09-12T14:44:28.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:28.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj105 td/osd-backfill-space/datafile 2020-09-12T14:44:28.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:28.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj105 td/osd-backfill-space/datafile 2020-09-12T14:44:29.038 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:29.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj105 td/osd-backfill-space/datafile 2020-09-12T14:44:29.113 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:29.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj105 td/osd-backfill-space/datafile 2020-09-12T14:44:29.191 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:29.191 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj105 td/osd-backfill-space/datafile 2020-09-12T14:44:29.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:29.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj105 td/osd-backfill-space/datafile 2020-09-12T14:44:29.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:29.348 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:29.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:29.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj106 td/osd-backfill-space/datafile 2020-09-12T14:44:29.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:29.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj106 td/osd-backfill-space/datafile 2020-09-12T14:44:29.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:29.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj106 td/osd-backfill-space/datafile 2020-09-12T14:44:29.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:29.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj106 td/osd-backfill-space/datafile 2020-09-12T14:44:29.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:29.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj106 td/osd-backfill-space/datafile 2020-09-12T14:44:29.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:29.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj106 td/osd-backfill-space/datafile 2020-09-12T14:44:29.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:29.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj106 td/osd-backfill-space/datafile 2020-09-12T14:44:29.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:29.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj106 td/osd-backfill-space/datafile 2020-09-12T14:44:30.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:30.027 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:30.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:30.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj107 td/osd-backfill-space/datafile 2020-09-12T14:44:30.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:30.106 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj107 td/osd-backfill-space/datafile 2020-09-12T14:44:30.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:30.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj107 td/osd-backfill-space/datafile 2020-09-12T14:44:30.261 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:30.261 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj107 td/osd-backfill-space/datafile 2020-09-12T14:44:30.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:30.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj107 td/osd-backfill-space/datafile 2020-09-12T14:44:30.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:30.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj107 td/osd-backfill-space/datafile 2020-09-12T14:44:30.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:30.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj107 td/osd-backfill-space/datafile 2020-09-12T14:44:30.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:30.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj107 td/osd-backfill-space/datafile 2020-09-12T14:44:30.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:30.638 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:30.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:30.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj108 td/osd-backfill-space/datafile 2020-09-12T14:44:30.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:30.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj108 td/osd-backfill-space/datafile 2020-09-12T14:44:30.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:30.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj108 td/osd-backfill-space/datafile 2020-09-12T14:44:30.864 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:30.864 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj108 td/osd-backfill-space/datafile 2020-09-12T14:44:30.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:30.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj108 td/osd-backfill-space/datafile 2020-09-12T14:44:31.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:31.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj108 td/osd-backfill-space/datafile 2020-09-12T14:44:31.090 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:31.091 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj108 td/osd-backfill-space/datafile 2020-09-12T14:44:31.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:31.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj108 td/osd-backfill-space/datafile 2020-09-12T14:44:31.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:31.250 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:31.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:31.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj109 td/osd-backfill-space/datafile 2020-09-12T14:44:31.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:31.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj109 td/osd-backfill-space/datafile 2020-09-12T14:44:31.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:31.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj109 td/osd-backfill-space/datafile 2020-09-12T14:44:31.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:31.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj109 td/osd-backfill-space/datafile 2020-09-12T14:44:31.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:31.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj109 td/osd-backfill-space/datafile 2020-09-12T14:44:31.668 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:31.669 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj109 td/osd-backfill-space/datafile 2020-09-12T14:44:31.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:31.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj109 td/osd-backfill-space/datafile 2020-09-12T14:44:31.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:31.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj109 td/osd-backfill-space/datafile 2020-09-12T14:44:31.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:31.938 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:31.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:31.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj110 td/osd-backfill-space/datafile 2020-09-12T14:44:32.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:32.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj110 td/osd-backfill-space/datafile 2020-09-12T14:44:32.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:32.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj110 td/osd-backfill-space/datafile 2020-09-12T14:44:32.222 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:32.222 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj110 td/osd-backfill-space/datafile 2020-09-12T14:44:32.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:32.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj110 td/osd-backfill-space/datafile 2020-09-12T14:44:32.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:32.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj110 td/osd-backfill-space/datafile 2020-09-12T14:44:32.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:32.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj110 td/osd-backfill-space/datafile 2020-09-12T14:44:32.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:32.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj110 td/osd-backfill-space/datafile 2020-09-12T14:44:32.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:32.908 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:32.908 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:32.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj111 td/osd-backfill-space/datafile 2020-09-12T14:44:32.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:32.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj111 td/osd-backfill-space/datafile 2020-09-12T14:44:33.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:33.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj111 td/osd-backfill-space/datafile 2020-09-12T14:44:33.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:33.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj111 td/osd-backfill-space/datafile 2020-09-12T14:44:33.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:33.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj111 td/osd-backfill-space/datafile 2020-09-12T14:44:33.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:33.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj111 td/osd-backfill-space/datafile 2020-09-12T14:44:33.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:33.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj111 td/osd-backfill-space/datafile 2020-09-12T14:44:33.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:33.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj111 td/osd-backfill-space/datafile 2020-09-12T14:44:33.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:33.586 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:33.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:33.587 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj112 td/osd-backfill-space/datafile 2020-09-12T14:44:33.653 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:33.653 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj112 td/osd-backfill-space/datafile 2020-09-12T14:44:33.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:33.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj112 td/osd-backfill-space/datafile 2020-09-12T14:44:33.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:33.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj112 td/osd-backfill-space/datafile 2020-09-12T14:44:33.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:33.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj112 td/osd-backfill-space/datafile 2020-09-12T14:44:33.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:33.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj112 td/osd-backfill-space/datafile 2020-09-12T14:44:34.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:34.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj112 td/osd-backfill-space/datafile 2020-09-12T14:44:34.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:34.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj112 td/osd-backfill-space/datafile 2020-09-12T14:44:34.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:34.215 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:34.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:34.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj113 td/osd-backfill-space/datafile 2020-09-12T14:44:34.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:34.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj113 td/osd-backfill-space/datafile 2020-09-12T14:44:34.365 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:34.365 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj113 td/osd-backfill-space/datafile 2020-09-12T14:44:34.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:34.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj113 td/osd-backfill-space/datafile 2020-09-12T14:44:34.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:34.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj113 td/osd-backfill-space/datafile 2020-09-12T14:44:34.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:34.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj113 td/osd-backfill-space/datafile 2020-09-12T14:44:34.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:34.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj113 td/osd-backfill-space/datafile 2020-09-12T14:44:34.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:34.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj113 td/osd-backfill-space/datafile 2020-09-12T14:44:34.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:34.835 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:34.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:34.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj114 td/osd-backfill-space/datafile 2020-09-12T14:44:34.914 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:34.914 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj114 td/osd-backfill-space/datafile 2020-09-12T14:44:34.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:34.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj114 td/osd-backfill-space/datafile 2020-09-12T14:44:35.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:35.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj114 td/osd-backfill-space/datafile 2020-09-12T14:44:35.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:35.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj114 td/osd-backfill-space/datafile 2020-09-12T14:44:35.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:35.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj114 td/osd-backfill-space/datafile 2020-09-12T14:44:35.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:35.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj114 td/osd-backfill-space/datafile 2020-09-12T14:44:35.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:35.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj114 td/osd-backfill-space/datafile 2020-09-12T14:44:35.457 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:35.458 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:35.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:35.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj115 td/osd-backfill-space/datafile 2020-09-12T14:44:35.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:35.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj115 td/osd-backfill-space/datafile 2020-09-12T14:44:35.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:35.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj115 td/osd-backfill-space/datafile 2020-09-12T14:44:35.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:35.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj115 td/osd-backfill-space/datafile 2020-09-12T14:44:35.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:35.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj115 td/osd-backfill-space/datafile 2020-09-12T14:44:35.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:35.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj115 td/osd-backfill-space/datafile 2020-09-12T14:44:35.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:35.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj115 td/osd-backfill-space/datafile 2020-09-12T14:44:36.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:36.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj115 td/osd-backfill-space/datafile 2020-09-12T14:44:36.116 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:36.116 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:36.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:36.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj116 td/osd-backfill-space/datafile 2020-09-12T14:44:36.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:36.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj116 td/osd-backfill-space/datafile 2020-09-12T14:44:36.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:36.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj116 td/osd-backfill-space/datafile 2020-09-12T14:44:36.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:36.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj116 td/osd-backfill-space/datafile 2020-09-12T14:44:36.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:36.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj116 td/osd-backfill-space/datafile 2020-09-12T14:44:36.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:36.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj116 td/osd-backfill-space/datafile 2020-09-12T14:44:36.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:36.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj116 td/osd-backfill-space/datafile 2020-09-12T14:44:36.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:36.653 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj116 td/osd-backfill-space/datafile 2020-09-12T14:44:36.728 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:36.729 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:36.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:36.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj117 td/osd-backfill-space/datafile 2020-09-12T14:44:36.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:36.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj117 td/osd-backfill-space/datafile 2020-09-12T14:44:36.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:36.887 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj117 td/osd-backfill-space/datafile 2020-09-12T14:44:36.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:36.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj117 td/osd-backfill-space/datafile 2020-09-12T14:44:37.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:37.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj117 td/osd-backfill-space/datafile 2020-09-12T14:44:37.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:37.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj117 td/osd-backfill-space/datafile 2020-09-12T14:44:37.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:37.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj117 td/osd-backfill-space/datafile 2020-09-12T14:44:37.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:37.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj117 td/osd-backfill-space/datafile 2020-09-12T14:44:37.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:37.390 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:37.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:37.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj118 td/osd-backfill-space/datafile 2020-09-12T14:44:37.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:37.507 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj118 td/osd-backfill-space/datafile 2020-09-12T14:44:37.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:37.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj118 td/osd-backfill-space/datafile 2020-09-12T14:44:37.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:37.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj118 td/osd-backfill-space/datafile 2020-09-12T14:44:37.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:37.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj118 td/osd-backfill-space/datafile 2020-09-12T14:44:37.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:37.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj118 td/osd-backfill-space/datafile 2020-09-12T14:44:37.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:37.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj118 td/osd-backfill-space/datafile 2020-09-12T14:44:37.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:37.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj118 td/osd-backfill-space/datafile 2020-09-12T14:44:38.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:38.060 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:38.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:38.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj119 td/osd-backfill-space/datafile 2020-09-12T14:44:38.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:38.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj119 td/osd-backfill-space/datafile 2020-09-12T14:44:38.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:38.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj119 td/osd-backfill-space/datafile 2020-09-12T14:44:38.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:38.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj119 td/osd-backfill-space/datafile 2020-09-12T14:44:38.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:38.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj119 td/osd-backfill-space/datafile 2020-09-12T14:44:38.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:38.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj119 td/osd-backfill-space/datafile 2020-09-12T14:44:38.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:38.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj119 td/osd-backfill-space/datafile 2020-09-12T14:44:38.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:38.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj119 td/osd-backfill-space/datafile 2020-09-12T14:44:38.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:38.789 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:38.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:38.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj120 td/osd-backfill-space/datafile 2020-09-12T14:44:38.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:38.864 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj120 td/osd-backfill-space/datafile 2020-09-12T14:44:38.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:38.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj120 td/osd-backfill-space/datafile 2020-09-12T14:44:39.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:39.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj120 td/osd-backfill-space/datafile 2020-09-12T14:44:39.106 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:39.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj120 td/osd-backfill-space/datafile 2020-09-12T14:44:39.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:39.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj120 td/osd-backfill-space/datafile 2020-09-12T14:44:39.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:39.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj120 td/osd-backfill-space/datafile 2020-09-12T14:44:39.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:39.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj120 td/osd-backfill-space/datafile 2020-09-12T14:44:39.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:39.408 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:39.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:39.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj121 td/osd-backfill-space/datafile 2020-09-12T14:44:39.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:39.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj121 td/osd-backfill-space/datafile 2020-09-12T14:44:39.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:39.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj121 td/osd-backfill-space/datafile 2020-09-12T14:44:39.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:39.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj121 td/osd-backfill-space/datafile 2020-09-12T14:44:39.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:39.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj121 td/osd-backfill-space/datafile 2020-09-12T14:44:39.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:39.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj121 td/osd-backfill-space/datafile 2020-09-12T14:44:39.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:39.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj121 td/osd-backfill-space/datafile 2020-09-12T14:44:40.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:40.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj121 td/osd-backfill-space/datafile 2020-09-12T14:44:40.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:40.112 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:40.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:40.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj122 td/osd-backfill-space/datafile 2020-09-12T14:44:40.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:40.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj122 td/osd-backfill-space/datafile 2020-09-12T14:44:40.262 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:40.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj122 td/osd-backfill-space/datafile 2020-09-12T14:44:40.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:40.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj122 td/osd-backfill-space/datafile 2020-09-12T14:44:40.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:40.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj122 td/osd-backfill-space/datafile 2020-09-12T14:44:40.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:40.489 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj122 td/osd-backfill-space/datafile 2020-09-12T14:44:40.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:40.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj122 td/osd-backfill-space/datafile 2020-09-12T14:44:40.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:40.640 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj122 td/osd-backfill-space/datafile 2020-09-12T14:44:40.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:40.715 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:40.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:40.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj123 td/osd-backfill-space/datafile 2020-09-12T14:44:40.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:40.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj123 td/osd-backfill-space/datafile 2020-09-12T14:44:40.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:40.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj123 td/osd-backfill-space/datafile 2020-09-12T14:44:40.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:40.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj123 td/osd-backfill-space/datafile 2020-09-12T14:44:41.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:41.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj123 td/osd-backfill-space/datafile 2020-09-12T14:44:41.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:41.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj123 td/osd-backfill-space/datafile 2020-09-12T14:44:41.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:41.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj123 td/osd-backfill-space/datafile 2020-09-12T14:44:41.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:41.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj123 td/osd-backfill-space/datafile 2020-09-12T14:44:41.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:41.335 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:41.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:41.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj124 td/osd-backfill-space/datafile 2020-09-12T14:44:41.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:41.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj124 td/osd-backfill-space/datafile 2020-09-12T14:44:41.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:41.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj124 td/osd-backfill-space/datafile 2020-09-12T14:44:41.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:41.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj124 td/osd-backfill-space/datafile 2020-09-12T14:44:41.678 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:41.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj124 td/osd-backfill-space/datafile 2020-09-12T14:44:41.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:41.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj124 td/osd-backfill-space/datafile 2020-09-12T14:44:41.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:41.863 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj124 td/osd-backfill-space/datafile 2020-09-12T14:44:41.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:41.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj124 td/osd-backfill-space/datafile 2020-09-12T14:44:42.014 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:42.014 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:42.014 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:42.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj125 td/osd-backfill-space/datafile 2020-09-12T14:44:42.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:42.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj125 td/osd-backfill-space/datafile 2020-09-12T14:44:42.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:42.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj125 td/osd-backfill-space/datafile 2020-09-12T14:44:42.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:42.265 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj125 td/osd-backfill-space/datafile 2020-09-12T14:44:42.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:42.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj125 td/osd-backfill-space/datafile 2020-09-12T14:44:42.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:42.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj125 td/osd-backfill-space/datafile 2020-09-12T14:44:42.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:42.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj125 td/osd-backfill-space/datafile 2020-09-12T14:44:42.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:42.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj125 td/osd-backfill-space/datafile 2020-09-12T14:44:42.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:42.651 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:42.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:42.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj126 td/osd-backfill-space/datafile 2020-09-12T14:44:42.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:42.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj126 td/osd-backfill-space/datafile 2020-09-12T14:44:42.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:42.810 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj126 td/osd-backfill-space/datafile 2020-09-12T14:44:42.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:42.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj126 td/osd-backfill-space/datafile 2020-09-12T14:44:42.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:42.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj126 td/osd-backfill-space/datafile 2020-09-12T14:44:43.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:43.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj126 td/osd-backfill-space/datafile 2020-09-12T14:44:43.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:43.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj126 td/osd-backfill-space/datafile 2020-09-12T14:44:43.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:43.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj126 td/osd-backfill-space/datafile 2020-09-12T14:44:43.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:43.324 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:43.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:43.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj127 td/osd-backfill-space/datafile 2020-09-12T14:44:43.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:43.404 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj127 td/osd-backfill-space/datafile 2020-09-12T14:44:43.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:43.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj127 td/osd-backfill-space/datafile 2020-09-12T14:44:43.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:43.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj127 td/osd-backfill-space/datafile 2020-09-12T14:44:43.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:43.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj127 td/osd-backfill-space/datafile 2020-09-12T14:44:43.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:43.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj127 td/osd-backfill-space/datafile 2020-09-12T14:44:43.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:43.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj127 td/osd-backfill-space/datafile 2020-09-12T14:44:43.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:43.966 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj127 td/osd-backfill-space/datafile 2020-09-12T14:44:44.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:44.041 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:44.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:44.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj128 td/osd-backfill-space/datafile 2020-09-12T14:44:44.116 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:44.116 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj128 td/osd-backfill-space/datafile 2020-09-12T14:44:44.191 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:44.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj128 td/osd-backfill-space/datafile 2020-09-12T14:44:44.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:44.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj128 td/osd-backfill-space/datafile 2020-09-12T14:44:44.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:44.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj128 td/osd-backfill-space/datafile 2020-09-12T14:44:44.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:44.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj128 td/osd-backfill-space/datafile 2020-09-12T14:44:44.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:44.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj128 td/osd-backfill-space/datafile 2020-09-12T14:44:44.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:44.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj128 td/osd-backfill-space/datafile 2020-09-12T14:44:44.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:44.636 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:44.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:44.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj129 td/osd-backfill-space/datafile 2020-09-12T14:44:44.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:44.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj129 td/osd-backfill-space/datafile 2020-09-12T14:44:44.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:44.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj129 td/osd-backfill-space/datafile 2020-09-12T14:44:44.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:44.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj129 td/osd-backfill-space/datafile 2020-09-12T14:44:44.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:44.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj129 td/osd-backfill-space/datafile 2020-09-12T14:44:45.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:45.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj129 td/osd-backfill-space/datafile 2020-09-12T14:44:45.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:45.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj129 td/osd-backfill-space/datafile 2020-09-12T14:44:45.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:45.172 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj129 td/osd-backfill-space/datafile 2020-09-12T14:44:45.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:45.256 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:45.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:45.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj130 td/osd-backfill-space/datafile 2020-09-12T14:44:45.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:45.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj130 td/osd-backfill-space/datafile 2020-09-12T14:44:45.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:45.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj130 td/osd-backfill-space/datafile 2020-09-12T14:44:45.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:45.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj130 td/osd-backfill-space/datafile 2020-09-12T14:44:45.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:45.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj130 td/osd-backfill-space/datafile 2020-09-12T14:44:45.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:45.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj130 td/osd-backfill-space/datafile 2020-09-12T14:44:45.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:45.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj130 td/osd-backfill-space/datafile 2020-09-12T14:44:45.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:45.867 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj130 td/osd-backfill-space/datafile 2020-09-12T14:44:45.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:45.943 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:45.943 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:45.943 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj131 td/osd-backfill-space/datafile 2020-09-12T14:44:46.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:46.018 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj131 td/osd-backfill-space/datafile 2020-09-12T14:44:46.093 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:46.093 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj131 td/osd-backfill-space/datafile 2020-09-12T14:44:46.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:46.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj131 td/osd-backfill-space/datafile 2020-09-12T14:44:46.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:46.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj131 td/osd-backfill-space/datafile 2020-09-12T14:44:46.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:46.328 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj131 td/osd-backfill-space/datafile 2020-09-12T14:44:46.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:46.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj131 td/osd-backfill-space/datafile 2020-09-12T14:44:46.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:46.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj131 td/osd-backfill-space/datafile 2020-09-12T14:44:46.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:46.554 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:46.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:46.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj132 td/osd-backfill-space/datafile 2020-09-12T14:44:46.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:46.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj132 td/osd-backfill-space/datafile 2020-09-12T14:44:46.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:46.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj132 td/osd-backfill-space/datafile 2020-09-12T14:44:46.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:46.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj132 td/osd-backfill-space/datafile 2020-09-12T14:44:46.864 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:46.864 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj132 td/osd-backfill-space/datafile 2020-09-12T14:44:46.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:46.939 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj132 td/osd-backfill-space/datafile 2020-09-12T14:44:47.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:47.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj132 td/osd-backfill-space/datafile 2020-09-12T14:44:47.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:47.140 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj132 td/osd-backfill-space/datafile 2020-09-12T14:44:47.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:47.216 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:47.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:47.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj133 td/osd-backfill-space/datafile 2020-09-12T14:44:47.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:47.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj133 td/osd-backfill-space/datafile 2020-09-12T14:44:47.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:47.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj133 td/osd-backfill-space/datafile 2020-09-12T14:44:47.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:47.484 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj133 td/osd-backfill-space/datafile 2020-09-12T14:44:47.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:47.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj133 td/osd-backfill-space/datafile 2020-09-12T14:44:47.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:47.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj133 td/osd-backfill-space/datafile 2020-09-12T14:44:47.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:47.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj133 td/osd-backfill-space/datafile 2020-09-12T14:44:47.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:47.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj133 td/osd-backfill-space/datafile 2020-09-12T14:44:47.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:47.903 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:47.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:47.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj134 td/osd-backfill-space/datafile 2020-09-12T14:44:47.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:47.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj134 td/osd-backfill-space/datafile 2020-09-12T14:44:48.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:48.054 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj134 td/osd-backfill-space/datafile 2020-09-12T14:44:48.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:48.129 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj134 td/osd-backfill-space/datafile 2020-09-12T14:44:48.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:48.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj134 td/osd-backfill-space/datafile 2020-09-12T14:44:48.288 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:48.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj134 td/osd-backfill-space/datafile 2020-09-12T14:44:48.363 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:48.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj134 td/osd-backfill-space/datafile 2020-09-12T14:44:48.439 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:48.439 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj134 td/osd-backfill-space/datafile 2020-09-12T14:44:48.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:48.523 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:48.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:48.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj135 td/osd-backfill-space/datafile 2020-09-12T14:44:48.598 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:48.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj135 td/osd-backfill-space/datafile 2020-09-12T14:44:48.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:48.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj135 td/osd-backfill-space/datafile 2020-09-12T14:44:48.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:48.749 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj135 td/osd-backfill-space/datafile 2020-09-12T14:44:48.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:48.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj135 td/osd-backfill-space/datafile 2020-09-12T14:44:48.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:48.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj135 td/osd-backfill-space/datafile 2020-09-12T14:44:48.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:48.975 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj135 td/osd-backfill-space/datafile 2020-09-12T14:44:49.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:49.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj135 td/osd-backfill-space/datafile 2020-09-12T14:44:49.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:49.118 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:49.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:49.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj136 td/osd-backfill-space/datafile 2020-09-12T14:44:49.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:49.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj136 td/osd-backfill-space/datafile 2020-09-12T14:44:49.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:49.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj136 td/osd-backfill-space/datafile 2020-09-12T14:44:49.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:49.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj136 td/osd-backfill-space/datafile 2020-09-12T14:44:49.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:49.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj136 td/osd-backfill-space/datafile 2020-09-12T14:44:49.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:49.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj136 td/osd-backfill-space/datafile 2020-09-12T14:44:49.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:49.621 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj136 td/osd-backfill-space/datafile 2020-09-12T14:44:49.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:49.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj136 td/osd-backfill-space/datafile 2020-09-12T14:44:49.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:49.813 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:49.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:49.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj137 td/osd-backfill-space/datafile 2020-09-12T14:44:49.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:49.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj137 td/osd-backfill-space/datafile 2020-09-12T14:44:49.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:49.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj137 td/osd-backfill-space/datafile 2020-09-12T14:44:50.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:50.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj137 td/osd-backfill-space/datafile 2020-09-12T14:44:50.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:50.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj137 td/osd-backfill-space/datafile 2020-09-12T14:44:50.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:50.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj137 td/osd-backfill-space/datafile 2020-09-12T14:44:50.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:50.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj137 td/osd-backfill-space/datafile 2020-09-12T14:44:50.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:50.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj137 td/osd-backfill-space/datafile 2020-09-12T14:44:50.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:50.425 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:50.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:50.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj138 td/osd-backfill-space/datafile 2020-09-12T14:44:50.499 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:50.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj138 td/osd-backfill-space/datafile 2020-09-12T14:44:50.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:50.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj138 td/osd-backfill-space/datafile 2020-09-12T14:44:50.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:50.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj138 td/osd-backfill-space/datafile 2020-09-12T14:44:50.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:50.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj138 td/osd-backfill-space/datafile 2020-09-12T14:44:50.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:50.801 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj138 td/osd-backfill-space/datafile 2020-09-12T14:44:50.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:50.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj138 td/osd-backfill-space/datafile 2020-09-12T14:44:50.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:50.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj138 td/osd-backfill-space/datafile 2020-09-12T14:44:51.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:51.028 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:51.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:51.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj139 td/osd-backfill-space/datafile 2020-09-12T14:44:51.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:51.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj139 td/osd-backfill-space/datafile 2020-09-12T14:44:51.178 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:51.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj139 td/osd-backfill-space/datafile 2020-09-12T14:44:51.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:51.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj139 td/osd-backfill-space/datafile 2020-09-12T14:44:51.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:51.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj139 td/osd-backfill-space/datafile 2020-09-12T14:44:51.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:51.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj139 td/osd-backfill-space/datafile 2020-09-12T14:44:51.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:51.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj139 td/osd-backfill-space/datafile 2020-09-12T14:44:51.597 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:51.598 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj139 td/osd-backfill-space/datafile 2020-09-12T14:44:51.673 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:51.674 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:51.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:51.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj140 td/osd-backfill-space/datafile 2020-09-12T14:44:51.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:51.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj140 td/osd-backfill-space/datafile 2020-09-12T14:44:51.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:51.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj140 td/osd-backfill-space/datafile 2020-09-12T14:44:51.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:51.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj140 td/osd-backfill-space/datafile 2020-09-12T14:44:52.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:52.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj140 td/osd-backfill-space/datafile 2020-09-12T14:44:52.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:52.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj140 td/osd-backfill-space/datafile 2020-09-12T14:44:52.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:52.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj140 td/osd-backfill-space/datafile 2020-09-12T14:44:52.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:52.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj140 td/osd-backfill-space/datafile 2020-09-12T14:44:52.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:52.352 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:52.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:52.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj141 td/osd-backfill-space/datafile 2020-09-12T14:44:52.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:52.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj141 td/osd-backfill-space/datafile 2020-09-12T14:44:52.510 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:52.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj141 td/osd-backfill-space/datafile 2020-09-12T14:44:52.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:52.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj141 td/osd-backfill-space/datafile 2020-09-12T14:44:52.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:52.670 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj141 td/osd-backfill-space/datafile 2020-09-12T14:44:52.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:52.745 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj141 td/osd-backfill-space/datafile 2020-09-12T14:44:52.820 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:52.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj141 td/osd-backfill-space/datafile 2020-09-12T14:44:52.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:52.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj141 td/osd-backfill-space/datafile 2020-09-12T14:44:52.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:52.980 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:52.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:52.981 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj142 td/osd-backfill-space/datafile 2020-09-12T14:44:53.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:53.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj142 td/osd-backfill-space/datafile 2020-09-12T14:44:53.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:53.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj142 td/osd-backfill-space/datafile 2020-09-12T14:44:53.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:53.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj142 td/osd-backfill-space/datafile 2020-09-12T14:44:53.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:53.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj142 td/osd-backfill-space/datafile 2020-09-12T14:44:53.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:53.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj142 td/osd-backfill-space/datafile 2020-09-12T14:44:53.482 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:53.482 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj142 td/osd-backfill-space/datafile 2020-09-12T14:44:53.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:53.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj142 td/osd-backfill-space/datafile 2020-09-12T14:44:53.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:53.785 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:53.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:53.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj143 td/osd-backfill-space/datafile 2020-09-12T14:44:53.859 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:53.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj143 td/osd-backfill-space/datafile 2020-09-12T14:44:53.934 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:53.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj143 td/osd-backfill-space/datafile 2020-09-12T14:44:54.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:54.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj143 td/osd-backfill-space/datafile 2020-09-12T14:44:54.085 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:54.085 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj143 td/osd-backfill-space/datafile 2020-09-12T14:44:54.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:54.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj143 td/osd-backfill-space/datafile 2020-09-12T14:44:54.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:54.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj143 td/osd-backfill-space/datafile 2020-09-12T14:44:54.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:54.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj143 td/osd-backfill-space/datafile 2020-09-12T14:44:54.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:54.404 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:54.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:54.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj144 td/osd-backfill-space/datafile 2020-09-12T14:44:54.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:54.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj144 td/osd-backfill-space/datafile 2020-09-12T14:44:54.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:54.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj144 td/osd-backfill-space/datafile 2020-09-12T14:44:54.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:54.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj144 td/osd-backfill-space/datafile 2020-09-12T14:44:54.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:54.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj144 td/osd-backfill-space/datafile 2020-09-12T14:44:54.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:54.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj144 td/osd-backfill-space/datafile 2020-09-12T14:44:54.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:54.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj144 td/osd-backfill-space/datafile 2020-09-12T14:44:54.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:54.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj144 td/osd-backfill-space/datafile 2020-09-12T14:44:55.032 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:55.033 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:55.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:55.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj145 td/osd-backfill-space/datafile 2020-09-12T14:44:55.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:55.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj145 td/osd-backfill-space/datafile 2020-09-12T14:44:55.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:55.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj145 td/osd-backfill-space/datafile 2020-09-12T14:44:55.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:55.258 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj145 td/osd-backfill-space/datafile 2020-09-12T14:44:55.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:55.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj145 td/osd-backfill-space/datafile 2020-09-12T14:44:55.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:55.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj145 td/osd-backfill-space/datafile 2020-09-12T14:44:55.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:55.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj145 td/osd-backfill-space/datafile 2020-09-12T14:44:55.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:55.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj145 td/osd-backfill-space/datafile 2020-09-12T14:44:55.677 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:55.677 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:55.678 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:55.678 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj146 td/osd-backfill-space/datafile 2020-09-12T14:44:55.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:55.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj146 td/osd-backfill-space/datafile 2020-09-12T14:44:55.861 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:55.862 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj146 td/osd-backfill-space/datafile 2020-09-12T14:44:55.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:55.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj146 td/osd-backfill-space/datafile 2020-09-12T14:44:56.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:56.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj146 td/osd-backfill-space/datafile 2020-09-12T14:44:56.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:56.088 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj146 td/osd-backfill-space/datafile 2020-09-12T14:44:56.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:56.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj146 td/osd-backfill-space/datafile 2020-09-12T14:44:56.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:56.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj146 td/osd-backfill-space/datafile 2020-09-12T14:44:56.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:56.314 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:56.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:56.315 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj147 td/osd-backfill-space/datafile 2020-09-12T14:44:56.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:56.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj147 td/osd-backfill-space/datafile 2020-09-12T14:44:56.464 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:56.465 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj147 td/osd-backfill-space/datafile 2020-09-12T14:44:56.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:56.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj147 td/osd-backfill-space/datafile 2020-09-12T14:44:56.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:56.616 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj147 td/osd-backfill-space/datafile 2020-09-12T14:44:56.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:56.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj147 td/osd-backfill-space/datafile 2020-09-12T14:44:56.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:56.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj147 td/osd-backfill-space/datafile 2020-09-12T14:44:56.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:56.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj147 td/osd-backfill-space/datafile 2020-09-12T14:44:56.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:56.917 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:56.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:56.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj148 td/osd-backfill-space/datafile 2020-09-12T14:44:56.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:56.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj148 td/osd-backfill-space/datafile 2020-09-12T14:44:57.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:57.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj148 td/osd-backfill-space/datafile 2020-09-12T14:44:57.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:57.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj148 td/osd-backfill-space/datafile 2020-09-12T14:44:57.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:57.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj148 td/osd-backfill-space/datafile 2020-09-12T14:44:57.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:57.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj148 td/osd-backfill-space/datafile 2020-09-12T14:44:57.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:57.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj148 td/osd-backfill-space/datafile 2020-09-12T14:44:57.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:57.529 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj148 td/osd-backfill-space/datafile 2020-09-12T14:44:57.604 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:57.605 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:57.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:57.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj149 td/osd-backfill-space/datafile 2020-09-12T14:44:57.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:57.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj149 td/osd-backfill-space/datafile 2020-09-12T14:44:57.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:57.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj149 td/osd-backfill-space/datafile 2020-09-12T14:44:57.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:57.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj149 td/osd-backfill-space/datafile 2020-09-12T14:44:57.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:57.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj149 td/osd-backfill-space/datafile 2020-09-12T14:44:58.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:58.032 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj149 td/osd-backfill-space/datafile 2020-09-12T14:44:58.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:58.107 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj149 td/osd-backfill-space/datafile 2020-09-12T14:44:58.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:58.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj149 td/osd-backfill-space/datafile 2020-09-12T14:44:58.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:58.266 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:58.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:58.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj150 td/osd-backfill-space/datafile 2020-09-12T14:44:58.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:58.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj150 td/osd-backfill-space/datafile 2020-09-12T14:44:58.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:58.417 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj150 td/osd-backfill-space/datafile 2020-09-12T14:44:58.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:58.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj150 td/osd-backfill-space/datafile 2020-09-12T14:44:58.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:58.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj150 td/osd-backfill-space/datafile 2020-09-12T14:44:58.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:58.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj150 td/osd-backfill-space/datafile 2020-09-12T14:44:58.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:58.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj150 td/osd-backfill-space/datafile 2020-09-12T14:44:58.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:58.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj150 td/osd-backfill-space/datafile 2020-09-12T14:44:58.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:58.927 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:58.928 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:58.928 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj151 td/osd-backfill-space/datafile 2020-09-12T14:44:59.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:59.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj151 td/osd-backfill-space/datafile 2020-09-12T14:44:59.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:59.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj151 td/osd-backfill-space/datafile 2020-09-12T14:44:59.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:59.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj151 td/osd-backfill-space/datafile 2020-09-12T14:44:59.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:59.246 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj151 td/osd-backfill-space/datafile 2020-09-12T14:44:59.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:59.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj151 td/osd-backfill-space/datafile 2020-09-12T14:44:59.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:59.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj151 td/osd-backfill-space/datafile 2020-09-12T14:44:59.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:59.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj151 td/osd-backfill-space/datafile 2020-09-12T14:44:59.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:44:59.582 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:44:59.583 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:59.583 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj152 td/osd-backfill-space/datafile 2020-09-12T14:44:59.664 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:59.665 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj152 td/osd-backfill-space/datafile 2020-09-12T14:44:59.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:59.751 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj152 td/osd-backfill-space/datafile 2020-09-12T14:44:59.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:59.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj152 td/osd-backfill-space/datafile 2020-09-12T14:44:59.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:44:59.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj152 td/osd-backfill-space/datafile 2020-09-12T14:45:00.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:00.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj152 td/osd-backfill-space/datafile 2020-09-12T14:45:00.083 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:00.083 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj152 td/osd-backfill-space/datafile 2020-09-12T14:45:00.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:00.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj152 td/osd-backfill-space/datafile 2020-09-12T14:45:00.236 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:00.236 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:00.236 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:00.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj153 td/osd-backfill-space/datafile 2020-09-12T14:45:00.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:00.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj153 td/osd-backfill-space/datafile 2020-09-12T14:45:00.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:00.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj153 td/osd-backfill-space/datafile 2020-09-12T14:45:00.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:00.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj153 td/osd-backfill-space/datafile 2020-09-12T14:45:00.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:00.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj153 td/osd-backfill-space/datafile 2020-09-12T14:45:00.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:00.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj153 td/osd-backfill-space/datafile 2020-09-12T14:45:00.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:00.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj153 td/osd-backfill-space/datafile 2020-09-12T14:45:00.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:00.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj153 td/osd-backfill-space/datafile 2020-09-12T14:45:00.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:00.855 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:00.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:00.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj154 td/osd-backfill-space/datafile 2020-09-12T14:45:00.929 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:00.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj154 td/osd-backfill-space/datafile 2020-09-12T14:45:01.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:01.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj154 td/osd-backfill-space/datafile 2020-09-12T14:45:01.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:01.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj154 td/osd-backfill-space/datafile 2020-09-12T14:45:01.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:01.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj154 td/osd-backfill-space/datafile 2020-09-12T14:45:01.231 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:01.231 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj154 td/osd-backfill-space/datafile 2020-09-12T14:45:01.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:01.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj154 td/osd-backfill-space/datafile 2020-09-12T14:45:01.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:01.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj154 td/osd-backfill-space/datafile 2020-09-12T14:45:01.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:01.491 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:01.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:01.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj155 td/osd-backfill-space/datafile 2020-09-12T14:45:01.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:01.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj155 td/osd-backfill-space/datafile 2020-09-12T14:45:01.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:01.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj155 td/osd-backfill-space/datafile 2020-09-12T14:45:01.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:01.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj155 td/osd-backfill-space/datafile 2020-09-12T14:45:01.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:01.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj155 td/osd-backfill-space/datafile 2020-09-12T14:45:01.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:01.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj155 td/osd-backfill-space/datafile 2020-09-12T14:45:01.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:01.980 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj155 td/osd-backfill-space/datafile 2020-09-12T14:45:02.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:02.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj155 td/osd-backfill-space/datafile 2020-09-12T14:45:02.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:02.136 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:02.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:02.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj156 td/osd-backfill-space/datafile 2020-09-12T14:45:02.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:02.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj156 td/osd-backfill-space/datafile 2020-09-12T14:45:02.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:02.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj156 td/osd-backfill-space/datafile 2020-09-12T14:45:02.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:02.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj156 td/osd-backfill-space/datafile 2020-09-12T14:45:02.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:02.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj156 td/osd-backfill-space/datafile 2020-09-12T14:45:02.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:02.547 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj156 td/osd-backfill-space/datafile 2020-09-12T14:45:02.622 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:02.622 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj156 td/osd-backfill-space/datafile 2020-09-12T14:45:02.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:02.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj156 td/osd-backfill-space/datafile 2020-09-12T14:45:02.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:02.773 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:02.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:02.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj157 td/osd-backfill-space/datafile 2020-09-12T14:45:02.848 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:02.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj157 td/osd-backfill-space/datafile 2020-09-12T14:45:02.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:02.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj157 td/osd-backfill-space/datafile 2020-09-12T14:45:02.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:02.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj157 td/osd-backfill-space/datafile 2020-09-12T14:45:03.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:03.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj157 td/osd-backfill-space/datafile 2020-09-12T14:45:03.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:03.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj157 td/osd-backfill-space/datafile 2020-09-12T14:45:03.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:03.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj157 td/osd-backfill-space/datafile 2020-09-12T14:45:03.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:03.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj157 td/osd-backfill-space/datafile 2020-09-12T14:45:03.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:03.393 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:03.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:03.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj158 td/osd-backfill-space/datafile 2020-09-12T14:45:03.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:03.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj158 td/osd-backfill-space/datafile 2020-09-12T14:45:03.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:03.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj158 td/osd-backfill-space/datafile 2020-09-12T14:45:03.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:03.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj158 td/osd-backfill-space/datafile 2020-09-12T14:45:03.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:03.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj158 td/osd-backfill-space/datafile 2020-09-12T14:45:04.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:04.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj158 td/osd-backfill-space/datafile 2020-09-12T14:45:04.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:04.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj158 td/osd-backfill-space/datafile 2020-09-12T14:45:04.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:04.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj158 td/osd-backfill-space/datafile 2020-09-12T14:45:04.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:04.389 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:04.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:04.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj159 td/osd-backfill-space/datafile 2020-09-12T14:45:04.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:04.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj159 td/osd-backfill-space/datafile 2020-09-12T14:45:04.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:04.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj159 td/osd-backfill-space/datafile 2020-09-12T14:45:04.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:04.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj159 td/osd-backfill-space/datafile 2020-09-12T14:45:04.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:04.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj159 td/osd-backfill-space/datafile 2020-09-12T14:45:04.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:04.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj159 td/osd-backfill-space/datafile 2020-09-12T14:45:04.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:04.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj159 td/osd-backfill-space/datafile 2020-09-12T14:45:04.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:04.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj159 td/osd-backfill-space/datafile 2020-09-12T14:45:05.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:05.058 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:05.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:05.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj160 td/osd-backfill-space/datafile 2020-09-12T14:45:05.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:05.134 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj160 td/osd-backfill-space/datafile 2020-09-12T14:45:05.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:05.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj160 td/osd-backfill-space/datafile 2020-09-12T14:45:05.284 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:05.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj160 td/osd-backfill-space/datafile 2020-09-12T14:45:05.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:05.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj160 td/osd-backfill-space/datafile 2020-09-12T14:45:05.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:05.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj160 td/osd-backfill-space/datafile 2020-09-12T14:45:05.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:05.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj160 td/osd-backfill-space/datafile 2020-09-12T14:45:05.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:05.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj160 td/osd-backfill-space/datafile 2020-09-12T14:45:05.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:05.703 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:05.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:05.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj161 td/osd-backfill-space/datafile 2020-09-12T14:45:05.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:05.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj161 td/osd-backfill-space/datafile 2020-09-12T14:45:05.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:05.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj161 td/osd-backfill-space/datafile 2020-09-12T14:45:05.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:05.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj161 td/osd-backfill-space/datafile 2020-09-12T14:45:06.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:06.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj161 td/osd-backfill-space/datafile 2020-09-12T14:45:06.122 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:06.122 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj161 td/osd-backfill-space/datafile 2020-09-12T14:45:06.197 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:06.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj161 td/osd-backfill-space/datafile 2020-09-12T14:45:06.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:06.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj161 td/osd-backfill-space/datafile 2020-09-12T14:45:06.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:06.365 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:06.366 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:06.366 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj162 td/osd-backfill-space/datafile 2020-09-12T14:45:06.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:06.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj162 td/osd-backfill-space/datafile 2020-09-12T14:45:06.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:06.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj162 td/osd-backfill-space/datafile 2020-09-12T14:45:06.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:06.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj162 td/osd-backfill-space/datafile 2020-09-12T14:45:06.666 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:06.667 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj162 td/osd-backfill-space/datafile 2020-09-12T14:45:06.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:06.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj162 td/osd-backfill-space/datafile 2020-09-12T14:45:06.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:06.826 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj162 td/osd-backfill-space/datafile 2020-09-12T14:45:06.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:06.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj162 td/osd-backfill-space/datafile 2020-09-12T14:45:06.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:06.977 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:06.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:06.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj163 td/osd-backfill-space/datafile 2020-09-12T14:45:07.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:07.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj163 td/osd-backfill-space/datafile 2020-09-12T14:45:07.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:07.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj163 td/osd-backfill-space/datafile 2020-09-12T14:45:07.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:07.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj163 td/osd-backfill-space/datafile 2020-09-12T14:45:07.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:07.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj163 td/osd-backfill-space/datafile 2020-09-12T14:45:07.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:07.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj163 td/osd-backfill-space/datafile 2020-09-12T14:45:07.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:07.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj163 td/osd-backfill-space/datafile 2020-09-12T14:45:07.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:07.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj163 td/osd-backfill-space/datafile 2020-09-12T14:45:07.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:07.656 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:07.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:07.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj164 td/osd-backfill-space/datafile 2020-09-12T14:45:07.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:07.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj164 td/osd-backfill-space/datafile 2020-09-12T14:45:07.848 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:07.848 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj164 td/osd-backfill-space/datafile 2020-09-12T14:45:07.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:07.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj164 td/osd-backfill-space/datafile 2020-09-12T14:45:08.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:08.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj164 td/osd-backfill-space/datafile 2020-09-12T14:45:08.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:08.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj164 td/osd-backfill-space/datafile 2020-09-12T14:45:08.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:08.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj164 td/osd-backfill-space/datafile 2020-09-12T14:45:08.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:08.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj164 td/osd-backfill-space/datafile 2020-09-12T14:45:08.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:08.325 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:08.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:08.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj165 td/osd-backfill-space/datafile 2020-09-12T14:45:08.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:08.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj165 td/osd-backfill-space/datafile 2020-09-12T14:45:08.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:08.468 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj165 td/osd-backfill-space/datafile 2020-09-12T14:45:08.551 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:08.551 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj165 td/osd-backfill-space/datafile 2020-09-12T14:45:08.626 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:08.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj165 td/osd-backfill-space/datafile 2020-09-12T14:45:08.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:08.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj165 td/osd-backfill-space/datafile 2020-09-12T14:45:08.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:08.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj165 td/osd-backfill-space/datafile 2020-09-12T14:45:08.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:08.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj165 td/osd-backfill-space/datafile 2020-09-12T14:45:08.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:08.920 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:08.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:08.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj166 td/osd-backfill-space/datafile 2020-09-12T14:45:09.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:09.245 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj166 td/osd-backfill-space/datafile 2020-09-12T14:45:09.246 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:09.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj166 td/osd-backfill-space/datafile 2020-09-12T14:45:09.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:09.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj166 td/osd-backfill-space/datafile 2020-09-12T14:45:09.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:09.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj166 td/osd-backfill-space/datafile 2020-09-12T14:45:09.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:09.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj166 td/osd-backfill-space/datafile 2020-09-12T14:45:09.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:09.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj166 td/osd-backfill-space/datafile 2020-09-12T14:45:09.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:09.649 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj166 td/osd-backfill-space/datafile 2020-09-12T14:45:09.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:09.724 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:09.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:09.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj167 td/osd-backfill-space/datafile 2020-09-12T14:45:09.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:09.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj167 td/osd-backfill-space/datafile 2020-09-12T14:45:09.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:09.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj167 td/osd-backfill-space/datafile 2020-09-12T14:45:09.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:09.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj167 td/osd-backfill-space/datafile 2020-09-12T14:45:10.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:10.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj167 td/osd-backfill-space/datafile 2020-09-12T14:45:10.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:10.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj167 td/osd-backfill-space/datafile 2020-09-12T14:45:10.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:10.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj167 td/osd-backfill-space/datafile 2020-09-12T14:45:10.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:10.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj167 td/osd-backfill-space/datafile 2020-09-12T14:45:10.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:10.336 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:10.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:10.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj168 td/osd-backfill-space/datafile 2020-09-12T14:45:10.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:10.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj168 td/osd-backfill-space/datafile 2020-09-12T14:45:10.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:10.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj168 td/osd-backfill-space/datafile 2020-09-12T14:45:10.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:10.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj168 td/osd-backfill-space/datafile 2020-09-12T14:45:10.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:10.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj168 td/osd-backfill-space/datafile 2020-09-12T14:45:10.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:10.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj168 td/osd-backfill-space/datafile 2020-09-12T14:45:10.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:10.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj168 td/osd-backfill-space/datafile 2020-09-12T14:45:10.871 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:10.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj168 td/osd-backfill-space/datafile 2020-09-12T14:45:10.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:10.947 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:10.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:10.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj169 td/osd-backfill-space/datafile 2020-09-12T14:45:11.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:11.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj169 td/osd-backfill-space/datafile 2020-09-12T14:45:11.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:11.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj169 td/osd-backfill-space/datafile 2020-09-12T14:45:11.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:11.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj169 td/osd-backfill-space/datafile 2020-09-12T14:45:11.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:11.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj169 td/osd-backfill-space/datafile 2020-09-12T14:45:11.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:11.333 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj169 td/osd-backfill-space/datafile 2020-09-12T14:45:11.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:11.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj169 td/osd-backfill-space/datafile 2020-09-12T14:45:11.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:11.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj169 td/osd-backfill-space/datafile 2020-09-12T14:45:11.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:11.601 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:11.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:11.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj170 td/osd-backfill-space/datafile 2020-09-12T14:45:11.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:11.676 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj170 td/osd-backfill-space/datafile 2020-09-12T14:45:11.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:11.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj170 td/osd-backfill-space/datafile 2020-09-12T14:45:11.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:11.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj170 td/osd-backfill-space/datafile 2020-09-12T14:45:11.944 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:11.944 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj170 td/osd-backfill-space/datafile 2020-09-12T14:45:12.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:12.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj170 td/osd-backfill-space/datafile 2020-09-12T14:45:12.094 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:12.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj170 td/osd-backfill-space/datafile 2020-09-12T14:45:12.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:12.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj170 td/osd-backfill-space/datafile 2020-09-12T14:45:12.272 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:12.273 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:12.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:12.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj171 td/osd-backfill-space/datafile 2020-09-12T14:45:12.365 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:12.365 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj171 td/osd-backfill-space/datafile 2020-09-12T14:45:12.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:12.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj171 td/osd-backfill-space/datafile 2020-09-12T14:45:12.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:12.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj171 td/osd-backfill-space/datafile 2020-09-12T14:45:12.591 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:12.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj171 td/osd-backfill-space/datafile 2020-09-12T14:45:12.666 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:12.667 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj171 td/osd-backfill-space/datafile 2020-09-12T14:45:12.742 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:12.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj171 td/osd-backfill-space/datafile 2020-09-12T14:45:12.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:12.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj171 td/osd-backfill-space/datafile 2020-09-12T14:45:12.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:12.910 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:12.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:12.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj172 td/osd-backfill-space/datafile 2020-09-12T14:45:12.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:12.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj172 td/osd-backfill-space/datafile 2020-09-12T14:45:13.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:13.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj172 td/osd-backfill-space/datafile 2020-09-12T14:45:13.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:13.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj172 td/osd-backfill-space/datafile 2020-09-12T14:45:13.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:13.211 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj172 td/osd-backfill-space/datafile 2020-09-12T14:45:13.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:13.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj172 td/osd-backfill-space/datafile 2020-09-12T14:45:13.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:13.362 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj172 td/osd-backfill-space/datafile 2020-09-12T14:45:13.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:13.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj172 td/osd-backfill-space/datafile 2020-09-12T14:45:13.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:13.561 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:13.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:13.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj173 td/osd-backfill-space/datafile 2020-09-12T14:45:13.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:13.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj173 td/osd-backfill-space/datafile 2020-09-12T14:45:13.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:13.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj173 td/osd-backfill-space/datafile 2020-09-12T14:45:13.823 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:13.824 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj173 td/osd-backfill-space/datafile 2020-09-12T14:45:13.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:13.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj173 td/osd-backfill-space/datafile 2020-09-12T14:45:13.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:14.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj173 td/osd-backfill-space/datafile 2020-09-12T14:45:14.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:14.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj173 td/osd-backfill-space/datafile 2020-09-12T14:45:14.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:14.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj173 td/osd-backfill-space/datafile 2020-09-12T14:45:14.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:14.237 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:14.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:14.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj174 td/osd-backfill-space/datafile 2020-09-12T14:45:14.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:14.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj174 td/osd-backfill-space/datafile 2020-09-12T14:45:14.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:14.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj174 td/osd-backfill-space/datafile 2020-09-12T14:45:14.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:14.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj174 td/osd-backfill-space/datafile 2020-09-12T14:45:14.585 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:14.586 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj174 td/osd-backfill-space/datafile 2020-09-12T14:45:14.660 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:14.661 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj174 td/osd-backfill-space/datafile 2020-09-12T14:45:14.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:14.736 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj174 td/osd-backfill-space/datafile 2020-09-12T14:45:14.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:14.811 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj174 td/osd-backfill-space/datafile 2020-09-12T14:45:14.895 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:14.896 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:14.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:14.896 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj175 td/osd-backfill-space/datafile 2020-09-12T14:45:14.970 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:14.971 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj175 td/osd-backfill-space/datafile 2020-09-12T14:45:15.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:15.046 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj175 td/osd-backfill-space/datafile 2020-09-12T14:45:15.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:15.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj175 td/osd-backfill-space/datafile 2020-09-12T14:45:15.196 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:15.197 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj175 td/osd-backfill-space/datafile 2020-09-12T14:45:15.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:15.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj175 td/osd-backfill-space/datafile 2020-09-12T14:45:15.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:15.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj175 td/osd-backfill-space/datafile 2020-09-12T14:45:15.431 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:15.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj175 td/osd-backfill-space/datafile 2020-09-12T14:45:15.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:15.532 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:15.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:15.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj176 td/osd-backfill-space/datafile 2020-09-12T14:45:15.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:15.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj176 td/osd-backfill-space/datafile 2020-09-12T14:45:15.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:15.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj176 td/osd-backfill-space/datafile 2020-09-12T14:45:15.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:15.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj176 td/osd-backfill-space/datafile 2020-09-12T14:45:15.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:15.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj176 td/osd-backfill-space/datafile 2020-09-12T14:45:15.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:15.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj176 td/osd-backfill-space/datafile 2020-09-12T14:45:16.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:16.035 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj176 td/osd-backfill-space/datafile 2020-09-12T14:45:16.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:16.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj176 td/osd-backfill-space/datafile 2020-09-12T14:45:16.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:16.194 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:16.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:16.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj177 td/osd-backfill-space/datafile 2020-09-12T14:45:16.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:16.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj177 td/osd-backfill-space/datafile 2020-09-12T14:45:16.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:16.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj177 td/osd-backfill-space/datafile 2020-09-12T14:45:16.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:16.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj177 td/osd-backfill-space/datafile 2020-09-12T14:45:16.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:16.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj177 td/osd-backfill-space/datafile 2020-09-12T14:45:16.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:16.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj177 td/osd-backfill-space/datafile 2020-09-12T14:45:16.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:16.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj177 td/osd-backfill-space/datafile 2020-09-12T14:45:16.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:16.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj177 td/osd-backfill-space/datafile 2020-09-12T14:45:16.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:16.797 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:16.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:16.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj178 td/osd-backfill-space/datafile 2020-09-12T14:45:16.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:16.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj178 td/osd-backfill-space/datafile 2020-09-12T14:45:16.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:16.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj178 td/osd-backfill-space/datafile 2020-09-12T14:45:17.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:17.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj178 td/osd-backfill-space/datafile 2020-09-12T14:45:17.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:17.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj178 td/osd-backfill-space/datafile 2020-09-12T14:45:17.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:17.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj178 td/osd-backfill-space/datafile 2020-09-12T14:45:17.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:17.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj178 td/osd-backfill-space/datafile 2020-09-12T14:45:17.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:17.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj178 td/osd-backfill-space/datafile 2020-09-12T14:45:17.500 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:17.501 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:17.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:17.502 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj179 td/osd-backfill-space/datafile 2020-09-12T14:45:17.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:17.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj179 td/osd-backfill-space/datafile 2020-09-12T14:45:17.660 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:17.660 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj179 td/osd-backfill-space/datafile 2020-09-12T14:45:17.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:17.743 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj179 td/osd-backfill-space/datafile 2020-09-12T14:45:17.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:17.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj179 td/osd-backfill-space/datafile 2020-09-12T14:45:17.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:17.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj179 td/osd-backfill-space/datafile 2020-09-12T14:45:17.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:17.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj179 td/osd-backfill-space/datafile 2020-09-12T14:45:18.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:18.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj179 td/osd-backfill-space/datafile 2020-09-12T14:45:18.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:18.146 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:18.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:18.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj180 td/osd-backfill-space/datafile 2020-09-12T14:45:18.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:18.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj180 td/osd-backfill-space/datafile 2020-09-12T14:45:18.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:18.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj180 td/osd-backfill-space/datafile 2020-09-12T14:45:18.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:18.388 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj180 td/osd-backfill-space/datafile 2020-09-12T14:45:18.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:18.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj180 td/osd-backfill-space/datafile 2020-09-12T14:45:18.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:18.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj180 td/osd-backfill-space/datafile 2020-09-12T14:45:18.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:18.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj180 td/osd-backfill-space/datafile 2020-09-12T14:45:18.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:18.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj180 td/osd-backfill-space/datafile 2020-09-12T14:45:18.782 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:18.783 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:18.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:18.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj181 td/osd-backfill-space/datafile 2020-09-12T14:45:18.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:18.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj181 td/osd-backfill-space/datafile 2020-09-12T14:45:18.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:18.942 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj181 td/osd-backfill-space/datafile 2020-09-12T14:45:19.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:19.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj181 td/osd-backfill-space/datafile 2020-09-12T14:45:19.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:19.092 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj181 td/osd-backfill-space/datafile 2020-09-12T14:45:19.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:19.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj181 td/osd-backfill-space/datafile 2020-09-12T14:45:19.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:19.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj181 td/osd-backfill-space/datafile 2020-09-12T14:45:19.318 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:19.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj181 td/osd-backfill-space/datafile 2020-09-12T14:45:19.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:19.394 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:19.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:19.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj182 td/osd-backfill-space/datafile 2020-09-12T14:45:19.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:19.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj182 td/osd-backfill-space/datafile 2020-09-12T14:45:19.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:19.595 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj182 td/osd-backfill-space/datafile 2020-09-12T14:45:19.678 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:19.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj182 td/osd-backfill-space/datafile 2020-09-12T14:45:19.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:19.780 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj182 td/osd-backfill-space/datafile 2020-09-12T14:45:19.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:19.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj182 td/osd-backfill-space/datafile 2020-09-12T14:45:19.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:19.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj182 td/osd-backfill-space/datafile 2020-09-12T14:45:20.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:20.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj182 td/osd-backfill-space/datafile 2020-09-12T14:45:20.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:20.098 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:20.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:20.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj183 td/osd-backfill-space/datafile 2020-09-12T14:45:20.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:20.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj183 td/osd-backfill-space/datafile 2020-09-12T14:45:20.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:20.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj183 td/osd-backfill-space/datafile 2020-09-12T14:45:20.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:20.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj183 td/osd-backfill-space/datafile 2020-09-12T14:45:20.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:20.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj183 td/osd-backfill-space/datafile 2020-09-12T14:45:20.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:20.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj183 td/osd-backfill-space/datafile 2020-09-12T14:45:20.558 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:20.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj183 td/osd-backfill-space/datafile 2020-09-12T14:45:20.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:20.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj183 td/osd-backfill-space/datafile 2020-09-12T14:45:20.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:20.710 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:20.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:20.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj184 td/osd-backfill-space/datafile 2020-09-12T14:45:20.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:20.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj184 td/osd-backfill-space/datafile 2020-09-12T14:45:20.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:20.869 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj184 td/osd-backfill-space/datafile 2020-09-12T14:45:20.944 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:20.944 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj184 td/osd-backfill-space/datafile 2020-09-12T14:45:21.019 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:21.020 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj184 td/osd-backfill-space/datafile 2020-09-12T14:45:21.086 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:21.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj184 td/osd-backfill-space/datafile 2020-09-12T14:45:21.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:21.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj184 td/osd-backfill-space/datafile 2020-09-12T14:45:21.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:21.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj184 td/osd-backfill-space/datafile 2020-09-12T14:45:21.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:21.321 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:21.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:21.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj185 td/osd-backfill-space/datafile 2020-09-12T14:45:21.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:21.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj185 td/osd-backfill-space/datafile 2020-09-12T14:45:21.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:21.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj185 td/osd-backfill-space/datafile 2020-09-12T14:45:21.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:21.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj185 td/osd-backfill-space/datafile 2020-09-12T14:45:21.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:21.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj185 td/osd-backfill-space/datafile 2020-09-12T14:45:21.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:21.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj185 td/osd-backfill-space/datafile 2020-09-12T14:45:21.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:21.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj185 td/osd-backfill-space/datafile 2020-09-12T14:45:21.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:21.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj185 td/osd-backfill-space/datafile 2020-09-12T14:45:22.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:22.050 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:22.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:22.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj186 td/osd-backfill-space/datafile 2020-09-12T14:45:22.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:22.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj186 td/osd-backfill-space/datafile 2020-09-12T14:45:22.200 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:22.201 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj186 td/osd-backfill-space/datafile 2020-09-12T14:45:22.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:22.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj186 td/osd-backfill-space/datafile 2020-09-12T14:45:22.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:22.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj186 td/osd-backfill-space/datafile 2020-09-12T14:45:22.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:22.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj186 td/osd-backfill-space/datafile 2020-09-12T14:45:22.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:22.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj186 td/osd-backfill-space/datafile 2020-09-12T14:45:22.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:22.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj186 td/osd-backfill-space/datafile 2020-09-12T14:45:22.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:22.704 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:22.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:22.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj187 td/osd-backfill-space/datafile 2020-09-12T14:45:22.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:22.779 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj187 td/osd-backfill-space/datafile 2020-09-12T14:45:22.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:22.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj187 td/osd-backfill-space/datafile 2020-09-12T14:45:22.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:22.938 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj187 td/osd-backfill-space/datafile 2020-09-12T14:45:23.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:23.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj187 td/osd-backfill-space/datafile 2020-09-12T14:45:23.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:23.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj187 td/osd-backfill-space/datafile 2020-09-12T14:45:23.172 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:23.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj187 td/osd-backfill-space/datafile 2020-09-12T14:45:23.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:23.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj187 td/osd-backfill-space/datafile 2020-09-12T14:45:23.340 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:23.340 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:23.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:23.341 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj188 td/osd-backfill-space/datafile 2020-09-12T14:45:23.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:23.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj188 td/osd-backfill-space/datafile 2020-09-12T14:45:23.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:23.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj188 td/osd-backfill-space/datafile 2020-09-12T14:45:23.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:23.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj188 td/osd-backfill-space/datafile 2020-09-12T14:45:23.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:23.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj188 td/osd-backfill-space/datafile 2020-09-12T14:45:23.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:23.809 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj188 td/osd-backfill-space/datafile 2020-09-12T14:45:23.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:23.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj188 td/osd-backfill-space/datafile 2020-09-12T14:45:23.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:23.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj188 td/osd-backfill-space/datafile 2020-09-12T14:45:24.035 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:24.036 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:24.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:24.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj189 td/osd-backfill-space/datafile 2020-09-12T14:45:24.119 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:24.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj189 td/osd-backfill-space/datafile 2020-09-12T14:45:24.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:24.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj189 td/osd-backfill-space/datafile 2020-09-12T14:45:24.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:24.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj189 td/osd-backfill-space/datafile 2020-09-12T14:45:24.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:24.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj189 td/osd-backfill-space/datafile 2020-09-12T14:45:24.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:24.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj189 td/osd-backfill-space/datafile 2020-09-12T14:45:24.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:24.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj189 td/osd-backfill-space/datafile 2020-09-12T14:45:24.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:24.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj189 td/osd-backfill-space/datafile 2020-09-12T14:45:24.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:24.689 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:24.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:24.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj190 td/osd-backfill-space/datafile 2020-09-12T14:45:24.897 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:24.898 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj190 td/osd-backfill-space/datafile 2020-09-12T14:45:24.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:24.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj190 td/osd-backfill-space/datafile 2020-09-12T14:45:25.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:25.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj190 td/osd-backfill-space/datafile 2020-09-12T14:45:25.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:25.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj190 td/osd-backfill-space/datafile 2020-09-12T14:45:25.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:25.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj190 td/osd-backfill-space/datafile 2020-09-12T14:45:25.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:25.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj190 td/osd-backfill-space/datafile 2020-09-12T14:45:25.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:25.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj190 td/osd-backfill-space/datafile 2020-09-12T14:45:25.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:25.426 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:25.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:25.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj191 td/osd-backfill-space/datafile 2020-09-12T14:45:25.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:25.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj191 td/osd-backfill-space/datafile 2020-09-12T14:45:25.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:25.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj191 td/osd-backfill-space/datafile 2020-09-12T14:45:25.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:25.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj191 td/osd-backfill-space/datafile 2020-09-12T14:45:25.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:25.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj191 td/osd-backfill-space/datafile 2020-09-12T14:45:25.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:25.911 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj191 td/osd-backfill-space/datafile 2020-09-12T14:45:25.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:25.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj191 td/osd-backfill-space/datafile 2020-09-12T14:45:26.062 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:26.062 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj191 td/osd-backfill-space/datafile 2020-09-12T14:45:26.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:26.147 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:26.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:26.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj192 td/osd-backfill-space/datafile 2020-09-12T14:45:26.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:26.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj192 td/osd-backfill-space/datafile 2020-09-12T14:45:26.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:26.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj192 td/osd-backfill-space/datafile 2020-09-12T14:45:26.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:26.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj192 td/osd-backfill-space/datafile 2020-09-12T14:45:26.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:26.456 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj192 td/osd-backfill-space/datafile 2020-09-12T14:45:26.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:26.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj192 td/osd-backfill-space/datafile 2020-09-12T14:45:26.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:26.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj192 td/osd-backfill-space/datafile 2020-09-12T14:45:26.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:26.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj192 td/osd-backfill-space/datafile 2020-09-12T14:45:26.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:26.775 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:26.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:26.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj193 td/osd-backfill-space/datafile 2020-09-12T14:45:26.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:26.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj193 td/osd-backfill-space/datafile 2020-09-12T14:45:26.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:26.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj193 td/osd-backfill-space/datafile 2020-09-12T14:45:27.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:27.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj193 td/osd-backfill-space/datafile 2020-09-12T14:45:27.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:27.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj193 td/osd-backfill-space/datafile 2020-09-12T14:45:27.151 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:27.151 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj193 td/osd-backfill-space/datafile 2020-09-12T14:45:27.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:27.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj193 td/osd-backfill-space/datafile 2020-09-12T14:45:27.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:27.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj193 td/osd-backfill-space/datafile 2020-09-12T14:45:27.444 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:27.445 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:27.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:27.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj194 td/osd-backfill-space/datafile 2020-09-12T14:45:27.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:27.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj194 td/osd-backfill-space/datafile 2020-09-12T14:45:27.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:27.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj194 td/osd-backfill-space/datafile 2020-09-12T14:45:27.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:27.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj194 td/osd-backfill-space/datafile 2020-09-12T14:45:27.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:27.822 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj194 td/osd-backfill-space/datafile 2020-09-12T14:45:27.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:27.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj194 td/osd-backfill-space/datafile 2020-09-12T14:45:27.989 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:27.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj194 td/osd-backfill-space/datafile 2020-09-12T14:45:28.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:28.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj194 td/osd-backfill-space/datafile 2020-09-12T14:45:28.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:28.149 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:28.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:28.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj195 td/osd-backfill-space/datafile 2020-09-12T14:45:28.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:28.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj195 td/osd-backfill-space/datafile 2020-09-12T14:45:28.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:28.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj195 td/osd-backfill-space/datafile 2020-09-12T14:45:28.396 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:28.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj195 td/osd-backfill-space/datafile 2020-09-12T14:45:28.467 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:28.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj195 td/osd-backfill-space/datafile 2020-09-12T14:45:28.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:28.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj195 td/osd-backfill-space/datafile 2020-09-12T14:45:28.617 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:28.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj195 td/osd-backfill-space/datafile 2020-09-12T14:45:28.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:28.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj195 td/osd-backfill-space/datafile 2020-09-12T14:45:28.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:28.777 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:28.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:28.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj196 td/osd-backfill-space/datafile 2020-09-12T14:45:28.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:28.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj196 td/osd-backfill-space/datafile 2020-09-12T14:45:28.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:28.928 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj196 td/osd-backfill-space/datafile 2020-09-12T14:45:29.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:29.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj196 td/osd-backfill-space/datafile 2020-09-12T14:45:29.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:29.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj196 td/osd-backfill-space/datafile 2020-09-12T14:45:29.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:29.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj196 td/osd-backfill-space/datafile 2020-09-12T14:45:29.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:29.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj196 td/osd-backfill-space/datafile 2020-09-12T14:45:29.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:29.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj196 td/osd-backfill-space/datafile 2020-09-12T14:45:29.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:29.413 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:29.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:29.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj197 td/osd-backfill-space/datafile 2020-09-12T14:45:29.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:29.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj197 td/osd-backfill-space/datafile 2020-09-12T14:45:29.598 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:29.598 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj197 td/osd-backfill-space/datafile 2020-09-12T14:45:29.673 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:29.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj197 td/osd-backfill-space/datafile 2020-09-12T14:45:29.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:29.774 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj197 td/osd-backfill-space/datafile 2020-09-12T14:45:29.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:29.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj197 td/osd-backfill-space/datafile 2020-09-12T14:45:29.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:29.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj197 td/osd-backfill-space/datafile 2020-09-12T14:45:30.066 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:30.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj197 td/osd-backfill-space/datafile 2020-09-12T14:45:30.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:30.143 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:30.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:30.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj198 td/osd-backfill-space/datafile 2020-09-12T14:45:30.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:30.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj198 td/osd-backfill-space/datafile 2020-09-12T14:45:30.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:30.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj198 td/osd-backfill-space/datafile 2020-09-12T14:45:30.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:30.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj198 td/osd-backfill-space/datafile 2020-09-12T14:45:30.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:30.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj198 td/osd-backfill-space/datafile 2020-09-12T14:45:30.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:30.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj198 td/osd-backfill-space/datafile 2020-09-12T14:45:30.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:30.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj198 td/osd-backfill-space/datafile 2020-09-12T14:45:30.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:30.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj198 td/osd-backfill-space/datafile 2020-09-12T14:45:30.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:30.771 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:30.771 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:30.771 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj199 td/osd-backfill-space/datafile 2020-09-12T14:45:30.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:30.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj199 td/osd-backfill-space/datafile 2020-09-12T14:45:30.929 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:30.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj199 td/osd-backfill-space/datafile 2020-09-12T14:45:31.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:31.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj199 td/osd-backfill-space/datafile 2020-09-12T14:45:31.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:31.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj199 td/osd-backfill-space/datafile 2020-09-12T14:45:31.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:31.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj199 td/osd-backfill-space/datafile 2020-09-12T14:45:31.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:31.240 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj199 td/osd-backfill-space/datafile 2020-09-12T14:45:31.315 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:31.315 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj199 td/osd-backfill-space/datafile 2020-09-12T14:45:31.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:31.392 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:31.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:31.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj200 td/osd-backfill-space/datafile 2020-09-12T14:45:31.508 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:31.509 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj200 td/osd-backfill-space/datafile 2020-09-12T14:45:31.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:31.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj200 td/osd-backfill-space/datafile 2020-09-12T14:45:31.667 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:31.668 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj200 td/osd-backfill-space/datafile 2020-09-12T14:45:31.775 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:31.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj200 td/osd-backfill-space/datafile 2020-09-12T14:45:31.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:31.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj200 td/osd-backfill-space/datafile 2020-09-12T14:45:31.926 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:31.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj200 td/osd-backfill-space/datafile 2020-09-12T14:45:32.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:32.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj200 td/osd-backfill-space/datafile 2020-09-12T14:45:32.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:32.077 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:32.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:32.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj201 td/osd-backfill-space/datafile 2020-09-12T14:45:32.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:32.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj201 td/osd-backfill-space/datafile 2020-09-12T14:45:32.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:32.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj201 td/osd-backfill-space/datafile 2020-09-12T14:45:32.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:32.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj201 td/osd-backfill-space/datafile 2020-09-12T14:45:32.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:32.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj201 td/osd-backfill-space/datafile 2020-09-12T14:45:32.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:32.522 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj201 td/osd-backfill-space/datafile 2020-09-12T14:45:32.597 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:32.597 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj201 td/osd-backfill-space/datafile 2020-09-12T14:45:32.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:32.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj201 td/osd-backfill-space/datafile 2020-09-12T14:45:32.765 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:32.765 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:32.766 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:32.766 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj202 td/osd-backfill-space/datafile 2020-09-12T14:45:32.848 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:32.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj202 td/osd-backfill-space/datafile 2020-09-12T14:45:32.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:32.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj202 td/osd-backfill-space/datafile 2020-09-12T14:45:32.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:32.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj202 td/osd-backfill-space/datafile 2020-09-12T14:45:33.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:33.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj202 td/osd-backfill-space/datafile 2020-09-12T14:45:33.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:33.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj202 td/osd-backfill-space/datafile 2020-09-12T14:45:33.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:33.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj202 td/osd-backfill-space/datafile 2020-09-12T14:45:33.319 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:33.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj202 td/osd-backfill-space/datafile 2020-09-12T14:45:33.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:33.393 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:33.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:33.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj203 td/osd-backfill-space/datafile 2020-09-12T14:45:33.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:33.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj203 td/osd-backfill-space/datafile 2020-09-12T14:45:33.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:33.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj203 td/osd-backfill-space/datafile 2020-09-12T14:45:33.669 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:33.669 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj203 td/osd-backfill-space/datafile 2020-09-12T14:45:33.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:33.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj203 td/osd-backfill-space/datafile 2020-09-12T14:45:33.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:33.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj203 td/osd-backfill-space/datafile 2020-09-12T14:45:33.929 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:33.929 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj203 td/osd-backfill-space/datafile 2020-09-12T14:45:34.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:34.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj203 td/osd-backfill-space/datafile 2020-09-12T14:45:34.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:34.080 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:34.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:34.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj204 td/osd-backfill-space/datafile 2020-09-12T14:45:34.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:34.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj204 td/osd-backfill-space/datafile 2020-09-12T14:45:34.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:34.239 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj204 td/osd-backfill-space/datafile 2020-09-12T14:45:34.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:34.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj204 td/osd-backfill-space/datafile 2020-09-12T14:45:34.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:34.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj204 td/osd-backfill-space/datafile 2020-09-12T14:45:34.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:34.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj204 td/osd-backfill-space/datafile 2020-09-12T14:45:34.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:34.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj204 td/osd-backfill-space/datafile 2020-09-12T14:45:34.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:34.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj204 td/osd-backfill-space/datafile 2020-09-12T14:45:34.725 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:34.725 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:34.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:34.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj205 td/osd-backfill-space/datafile 2020-09-12T14:45:34.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:34.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj205 td/osd-backfill-space/datafile 2020-09-12T14:45:34.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:34.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj205 td/osd-backfill-space/datafile 2020-09-12T14:45:35.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:35.043 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj205 td/osd-backfill-space/datafile 2020-09-12T14:45:35.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:35.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj205 td/osd-backfill-space/datafile 2020-09-12T14:45:35.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:35.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj205 td/osd-backfill-space/datafile 2020-09-12T14:45:35.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:35.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj205 td/osd-backfill-space/datafile 2020-09-12T14:45:35.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:35.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj205 td/osd-backfill-space/datafile 2020-09-12T14:45:35.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:35.704 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:35.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:35.705 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj206 td/osd-backfill-space/datafile 2020-09-12T14:45:35.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:35.821 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj206 td/osd-backfill-space/datafile 2020-09-12T14:45:35.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:35.905 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj206 td/osd-backfill-space/datafile 2020-09-12T14:45:35.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:35.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj206 td/osd-backfill-space/datafile 2020-09-12T14:45:36.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:36.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj206 td/osd-backfill-space/datafile 2020-09-12T14:45:36.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:36.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj206 td/osd-backfill-space/datafile 2020-09-12T14:45:36.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:36.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj206 td/osd-backfill-space/datafile 2020-09-12T14:45:36.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:36.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj206 td/osd-backfill-space/datafile 2020-09-12T14:45:36.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:36.374 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:36.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:36.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj207 td/osd-backfill-space/datafile 2020-09-12T14:45:36.457 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:36.457 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj207 td/osd-backfill-space/datafile 2020-09-12T14:45:36.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:36.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj207 td/osd-backfill-space/datafile 2020-09-12T14:45:36.616 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:36.617 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj207 td/osd-backfill-space/datafile 2020-09-12T14:45:36.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:36.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj207 td/osd-backfill-space/datafile 2020-09-12T14:45:36.767 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:36.767 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj207 td/osd-backfill-space/datafile 2020-09-12T14:45:36.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:36.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj207 td/osd-backfill-space/datafile 2020-09-12T14:45:36.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:36.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj207 td/osd-backfill-space/datafile 2020-09-12T14:45:36.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:36.994 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:36.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:36.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj208 td/osd-backfill-space/datafile 2020-09-12T14:45:37.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:37.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj208 td/osd-backfill-space/datafile 2020-09-12T14:45:37.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:37.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj208 td/osd-backfill-space/datafile 2020-09-12T14:45:37.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:37.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj208 td/osd-backfill-space/datafile 2020-09-12T14:45:37.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:37.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj208 td/osd-backfill-space/datafile 2020-09-12T14:45:37.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:37.371 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj208 td/osd-backfill-space/datafile 2020-09-12T14:45:37.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:37.480 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj208 td/osd-backfill-space/datafile 2020-09-12T14:45:37.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:37.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj208 td/osd-backfill-space/datafile 2020-09-12T14:45:37.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:37.656 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:37.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:37.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj209 td/osd-backfill-space/datafile 2020-09-12T14:45:37.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:37.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj209 td/osd-backfill-space/datafile 2020-09-12T14:45:37.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:37.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj209 td/osd-backfill-space/datafile 2020-09-12T14:45:37.915 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:37.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj209 td/osd-backfill-space/datafile 2020-09-12T14:45:37.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:37.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj209 td/osd-backfill-space/datafile 2020-09-12T14:45:38.066 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:38.066 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj209 td/osd-backfill-space/datafile 2020-09-12T14:45:38.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:38.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj209 td/osd-backfill-space/datafile 2020-09-12T14:45:38.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:38.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj209 td/osd-backfill-space/datafile 2020-09-12T14:45:38.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:38.293 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:38.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:38.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj210 td/osd-backfill-space/datafile 2020-09-12T14:45:38.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:38.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj210 td/osd-backfill-space/datafile 2020-09-12T14:45:38.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:38.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj210 td/osd-backfill-space/datafile 2020-09-12T14:45:38.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:38.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj210 td/osd-backfill-space/datafile 2020-09-12T14:45:38.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:38.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj210 td/osd-backfill-space/datafile 2020-09-12T14:45:38.677 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:38.678 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj210 td/osd-backfill-space/datafile 2020-09-12T14:45:38.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:38.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj210 td/osd-backfill-space/datafile 2020-09-12T14:45:38.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:38.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj210 td/osd-backfill-space/datafile 2020-09-12T14:45:38.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:38.913 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:38.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:38.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj211 td/osd-backfill-space/datafile 2020-09-12T14:45:38.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:38.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj211 td/osd-backfill-space/datafile 2020-09-12T14:45:39.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:39.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj211 td/osd-backfill-space/datafile 2020-09-12T14:45:39.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:39.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj211 td/osd-backfill-space/datafile 2020-09-12T14:45:39.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:39.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj211 td/osd-backfill-space/datafile 2020-09-12T14:45:39.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:39.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj211 td/osd-backfill-space/datafile 2020-09-12T14:45:39.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:39.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj211 td/osd-backfill-space/datafile 2020-09-12T14:45:39.468 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:39.469 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj211 td/osd-backfill-space/datafile 2020-09-12T14:45:39.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:39.560 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:39.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:39.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj212 td/osd-backfill-space/datafile 2020-09-12T14:45:39.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:39.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj212 td/osd-backfill-space/datafile 2020-09-12T14:45:39.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:39.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj212 td/osd-backfill-space/datafile 2020-09-12T14:45:39.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:39.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj212 td/osd-backfill-space/datafile 2020-09-12T14:45:39.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:39.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj212 td/osd-backfill-space/datafile 2020-09-12T14:45:39.968 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:39.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj212 td/osd-backfill-space/datafile 2020-09-12T14:45:40.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:40.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj212 td/osd-backfill-space/datafile 2020-09-12T14:45:40.119 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:40.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj212 td/osd-backfill-space/datafile 2020-09-12T14:45:40.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:40.195 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:40.196 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:40.196 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj213 td/osd-backfill-space/datafile 2020-09-12T14:45:40.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:40.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj213 td/osd-backfill-space/datafile 2020-09-12T14:45:40.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:40.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj213 td/osd-backfill-space/datafile 2020-09-12T14:45:40.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:40.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj213 td/osd-backfill-space/datafile 2020-09-12T14:45:40.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:40.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj213 td/osd-backfill-space/datafile 2020-09-12T14:45:40.606 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:40.606 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj213 td/osd-backfill-space/datafile 2020-09-12T14:45:40.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:40.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj213 td/osd-backfill-space/datafile 2020-09-12T14:45:40.864 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:40.864 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj213 td/osd-backfill-space/datafile 2020-09-12T14:45:40.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:40.948 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:40.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:40.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj214 td/osd-backfill-space/datafile 2020-09-12T14:45:41.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:41.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj214 td/osd-backfill-space/datafile 2020-09-12T14:45:41.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:41.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj214 td/osd-backfill-space/datafile 2020-09-12T14:45:41.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:41.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj214 td/osd-backfill-space/datafile 2020-09-12T14:45:41.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:41.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj214 td/osd-backfill-space/datafile 2020-09-12T14:45:41.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:41.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj214 td/osd-backfill-space/datafile 2020-09-12T14:45:41.417 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:41.417 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj214 td/osd-backfill-space/datafile 2020-09-12T14:45:41.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:41.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj214 td/osd-backfill-space/datafile 2020-09-12T14:45:41.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:41.610 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:41.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:41.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj215 td/osd-backfill-space/datafile 2020-09-12T14:45:41.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:41.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj215 td/osd-backfill-space/datafile 2020-09-12T14:45:41.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:41.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj215 td/osd-backfill-space/datafile 2020-09-12T14:45:41.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:41.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj215 td/osd-backfill-space/datafile 2020-09-12T14:45:41.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:41.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj215 td/osd-backfill-space/datafile 2020-09-12T14:45:42.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:42.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj215 td/osd-backfill-space/datafile 2020-09-12T14:45:42.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:42.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj215 td/osd-backfill-space/datafile 2020-09-12T14:45:42.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:42.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj215 td/osd-backfill-space/datafile 2020-09-12T14:45:42.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:42.255 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:42.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:42.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj216 td/osd-backfill-space/datafile 2020-09-12T14:45:42.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:42.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj216 td/osd-backfill-space/datafile 2020-09-12T14:45:42.447 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:42.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj216 td/osd-backfill-space/datafile 2020-09-12T14:45:42.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:42.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj216 td/osd-backfill-space/datafile 2020-09-12T14:45:42.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:42.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj216 td/osd-backfill-space/datafile 2020-09-12T14:45:42.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:42.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj216 td/osd-backfill-space/datafile 2020-09-12T14:45:42.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:42.750 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj216 td/osd-backfill-space/datafile 2020-09-12T14:45:42.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:42.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj216 td/osd-backfill-space/datafile 2020-09-12T14:45:42.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:42.909 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:42.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:42.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj217 td/osd-backfill-space/datafile 2020-09-12T14:45:42.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:42.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj217 td/osd-backfill-space/datafile 2020-09-12T14:45:43.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:43.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj217 td/osd-backfill-space/datafile 2020-09-12T14:45:43.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:43.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj217 td/osd-backfill-space/datafile 2020-09-12T14:45:43.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:43.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj217 td/osd-backfill-space/datafile 2020-09-12T14:45:43.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:43.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj217 td/osd-backfill-space/datafile 2020-09-12T14:45:43.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:43.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj217 td/osd-backfill-space/datafile 2020-09-12T14:45:43.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:43.454 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj217 td/osd-backfill-space/datafile 2020-09-12T14:45:43.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:43.555 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:43.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:43.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj218 td/osd-backfill-space/datafile 2020-09-12T14:45:43.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:43.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj218 td/osd-backfill-space/datafile 2020-09-12T14:45:43.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:43.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj218 td/osd-backfill-space/datafile 2020-09-12T14:45:43.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:43.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj218 td/osd-backfill-space/datafile 2020-09-12T14:45:43.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:43.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj218 td/osd-backfill-space/datafile 2020-09-12T14:45:43.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:43.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj218 td/osd-backfill-space/datafile 2020-09-12T14:45:44.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:44.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj218 td/osd-backfill-space/datafile 2020-09-12T14:45:44.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:44.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj218 td/osd-backfill-space/datafile 2020-09-12T14:45:44.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:44.250 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:44.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:44.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj219 td/osd-backfill-space/datafile 2020-09-12T14:45:44.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:44.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj219 td/osd-backfill-space/datafile 2020-09-12T14:45:44.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:44.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj219 td/osd-backfill-space/datafile 2020-09-12T14:45:44.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:44.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj219 td/osd-backfill-space/datafile 2020-09-12T14:45:44.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:44.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj219 td/osd-backfill-space/datafile 2020-09-12T14:45:44.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:44.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj219 td/osd-backfill-space/datafile 2020-09-12T14:45:44.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:44.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj219 td/osd-backfill-space/datafile 2020-09-12T14:45:44.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:44.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj219 td/osd-backfill-space/datafile 2020-09-12T14:45:44.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:44.886 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:44.887 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:44.887 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj220 td/osd-backfill-space/datafile 2020-09-12T14:45:44.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:44.962 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj220 td/osd-backfill-space/datafile 2020-09-12T14:45:45.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:45.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj220 td/osd-backfill-space/datafile 2020-09-12T14:45:45.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:45.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj220 td/osd-backfill-space/datafile 2020-09-12T14:45:45.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:45.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj220 td/osd-backfill-space/datafile 2020-09-12T14:45:45.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:45.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj220 td/osd-backfill-space/datafile 2020-09-12T14:45:45.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:45.339 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj220 td/osd-backfill-space/datafile 2020-09-12T14:45:45.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:45.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj220 td/osd-backfill-space/datafile 2020-09-12T14:45:45.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:45.532 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:45.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:45.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj221 td/osd-backfill-space/datafile 2020-09-12T14:45:45.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:45.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj221 td/osd-backfill-space/datafile 2020-09-12T14:45:45.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:45.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj221 td/osd-backfill-space/datafile 2020-09-12T14:45:45.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:45.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj221 td/osd-backfill-space/datafile 2020-09-12T14:45:45.933 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:45.934 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj221 td/osd-backfill-space/datafile 2020-09-12T14:45:46.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:46.001 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj221 td/osd-backfill-space/datafile 2020-09-12T14:45:46.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:46.076 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj221 td/osd-backfill-space/datafile 2020-09-12T14:45:46.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:46.160 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj221 td/osd-backfill-space/datafile 2020-09-12T14:45:46.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:46.235 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:46.236 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:46.236 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj222 td/osd-backfill-space/datafile 2020-09-12T14:45:46.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:46.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj222 td/osd-backfill-space/datafile 2020-09-12T14:45:46.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:46.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj222 td/osd-backfill-space/datafile 2020-09-12T14:45:46.494 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:46.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj222 td/osd-backfill-space/datafile 2020-09-12T14:45:46.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:46.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj222 td/osd-backfill-space/datafile 2020-09-12T14:45:46.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:46.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj222 td/osd-backfill-space/datafile 2020-09-12T14:45:46.737 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:46.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj222 td/osd-backfill-space/datafile 2020-09-12T14:45:46.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:46.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj222 td/osd-backfill-space/datafile 2020-09-12T14:45:46.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:46.889 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:46.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:46.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj223 td/osd-backfill-space/datafile 2020-09-12T14:45:46.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:46.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj223 td/osd-backfill-space/datafile 2020-09-12T14:45:47.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:47.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj223 td/osd-backfill-space/datafile 2020-09-12T14:45:47.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:47.131 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj223 td/osd-backfill-space/datafile 2020-09-12T14:45:47.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:47.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj223 td/osd-backfill-space/datafile 2020-09-12T14:45:47.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:47.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj223 td/osd-backfill-space/datafile 2020-09-12T14:45:47.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:47.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj223 td/osd-backfill-space/datafile 2020-09-12T14:45:47.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:47.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj223 td/osd-backfill-space/datafile 2020-09-12T14:45:47.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:47.593 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:47.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:47.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj224 td/osd-backfill-space/datafile 2020-09-12T14:45:47.667 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:47.668 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj224 td/osd-backfill-space/datafile 2020-09-12T14:45:47.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:47.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj224 td/osd-backfill-space/datafile 2020-09-12T14:45:47.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:47.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj224 td/osd-backfill-space/datafile 2020-09-12T14:45:47.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:47.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj224 td/osd-backfill-space/datafile 2020-09-12T14:45:48.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:48.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj224 td/osd-backfill-space/datafile 2020-09-12T14:45:48.094 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:48.095 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj224 td/osd-backfill-space/datafile 2020-09-12T14:45:48.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:48.170 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj224 td/osd-backfill-space/datafile 2020-09-12T14:45:48.253 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:48.254 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:48.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:48.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj225 td/osd-backfill-space/datafile 2020-09-12T14:45:48.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:48.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj225 td/osd-backfill-space/datafile 2020-09-12T14:45:48.437 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:48.438 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj225 td/osd-backfill-space/datafile 2020-09-12T14:45:48.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:48.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj225 td/osd-backfill-space/datafile 2020-09-12T14:45:48.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:48.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj225 td/osd-backfill-space/datafile 2020-09-12T14:45:48.664 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:48.664 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj225 td/osd-backfill-space/datafile 2020-09-12T14:45:48.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:48.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj225 td/osd-backfill-space/datafile 2020-09-12T14:45:48.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:48.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj225 td/osd-backfill-space/datafile 2020-09-12T14:45:48.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:48.899 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:48.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:48.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj226 td/osd-backfill-space/datafile 2020-09-12T14:45:48.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:48.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj226 td/osd-backfill-space/datafile 2020-09-12T14:45:49.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:49.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj226 td/osd-backfill-space/datafile 2020-09-12T14:45:49.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:49.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj226 td/osd-backfill-space/datafile 2020-09-12T14:45:49.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:49.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj226 td/osd-backfill-space/datafile 2020-09-12T14:45:49.284 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:49.284 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj226 td/osd-backfill-space/datafile 2020-09-12T14:45:49.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:49.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj226 td/osd-backfill-space/datafile 2020-09-12T14:45:49.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:49.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj226 td/osd-backfill-space/datafile 2020-09-12T14:45:49.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:49.536 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:49.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:49.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj227 td/osd-backfill-space/datafile 2020-09-12T14:45:49.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:49.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj227 td/osd-backfill-space/datafile 2020-09-12T14:45:49.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:49.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj227 td/osd-backfill-space/datafile 2020-09-12T14:45:49.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:49.787 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj227 td/osd-backfill-space/datafile 2020-09-12T14:45:49.871 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:49.914 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj227 td/osd-backfill-space/datafile 2020-09-12T14:45:49.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:49.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj227 td/osd-backfill-space/datafile 2020-09-12T14:45:50.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:50.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj227 td/osd-backfill-space/datafile 2020-09-12T14:45:50.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:50.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj227 td/osd-backfill-space/datafile 2020-09-12T14:45:50.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:50.184 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:50.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:50.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj228 td/osd-backfill-space/datafile 2020-09-12T14:45:50.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:50.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj228 td/osd-backfill-space/datafile 2020-09-12T14:45:50.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:50.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj228 td/osd-backfill-space/datafile 2020-09-12T14:45:50.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:50.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj228 td/osd-backfill-space/datafile 2020-09-12T14:45:50.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:50.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj228 td/osd-backfill-space/datafile 2020-09-12T14:45:50.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:50.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj228 td/osd-backfill-space/datafile 2020-09-12T14:45:50.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:50.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj228 td/osd-backfill-space/datafile 2020-09-12T14:45:50.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:50.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj228 td/osd-backfill-space/datafile 2020-09-12T14:45:50.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:50.792 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:50.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:50.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj229 td/osd-backfill-space/datafile 2020-09-12T14:45:50.878 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:50.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj229 td/osd-backfill-space/datafile 2020-09-12T14:45:50.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:50.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj229 td/osd-backfill-space/datafile 2020-09-12T14:45:51.035 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:51.035 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj229 td/osd-backfill-space/datafile 2020-09-12T14:45:51.110 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:51.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj229 td/osd-backfill-space/datafile 2020-09-12T14:45:51.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:51.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj229 td/osd-backfill-space/datafile 2020-09-12T14:45:51.261 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:51.261 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj229 td/osd-backfill-space/datafile 2020-09-12T14:45:51.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:51.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj229 td/osd-backfill-space/datafile 2020-09-12T14:45:51.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:51.420 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:51.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:51.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj230 td/osd-backfill-space/datafile 2020-09-12T14:45:51.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:51.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj230 td/osd-backfill-space/datafile 2020-09-12T14:45:51.621 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:51.621 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj230 td/osd-backfill-space/datafile 2020-09-12T14:45:51.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:51.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj230 td/osd-backfill-space/datafile 2020-09-12T14:45:51.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:51.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj230 td/osd-backfill-space/datafile 2020-09-12T14:45:51.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:51.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj230 td/osd-backfill-space/datafile 2020-09-12T14:45:51.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:51.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj230 td/osd-backfill-space/datafile 2020-09-12T14:45:52.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:52.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj230 td/osd-backfill-space/datafile 2020-09-12T14:45:52.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:52.142 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:52.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:52.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj231 td/osd-backfill-space/datafile 2020-09-12T14:45:52.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:52.216 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj231 td/osd-backfill-space/datafile 2020-09-12T14:45:52.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:52.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj231 td/osd-backfill-space/datafile 2020-09-12T14:45:52.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:52.367 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj231 td/osd-backfill-space/datafile 2020-09-12T14:45:52.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:52.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj231 td/osd-backfill-space/datafile 2020-09-12T14:45:52.551 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:52.551 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj231 td/osd-backfill-space/datafile 2020-09-12T14:45:52.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:52.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj231 td/osd-backfill-space/datafile 2020-09-12T14:45:52.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:52.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj231 td/osd-backfill-space/datafile 2020-09-12T14:45:52.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:52.795 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:52.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:52.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj232 td/osd-backfill-space/datafile 2020-09-12T14:45:52.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:52.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj232 td/osd-backfill-space/datafile 2020-09-12T14:45:52.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:52.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj232 td/osd-backfill-space/datafile 2020-09-12T14:45:53.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:53.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj232 td/osd-backfill-space/datafile 2020-09-12T14:45:53.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:53.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj232 td/osd-backfill-space/datafile 2020-09-12T14:45:53.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:53.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj232 td/osd-backfill-space/datafile 2020-09-12T14:45:53.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:53.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj232 td/osd-backfill-space/datafile 2020-09-12T14:45:53.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:53.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj232 td/osd-backfill-space/datafile 2020-09-12T14:45:53.414 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:53.415 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:53.415 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:53.416 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj233 td/osd-backfill-space/datafile 2020-09-12T14:45:53.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:53.540 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj233 td/osd-backfill-space/datafile 2020-09-12T14:45:53.615 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:53.616 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj233 td/osd-backfill-space/datafile 2020-09-12T14:45:53.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:53.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj233 td/osd-backfill-space/datafile 2020-09-12T14:45:53.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:53.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj233 td/osd-backfill-space/datafile 2020-09-12T14:45:53.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:53.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj233 td/osd-backfill-space/datafile 2020-09-12T14:45:53.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:53.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj233 td/osd-backfill-space/datafile 2020-09-12T14:45:54.059 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:54.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj233 td/osd-backfill-space/datafile 2020-09-12T14:45:54.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:54.136 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:54.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:54.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj234 td/osd-backfill-space/datafile 2020-09-12T14:45:54.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:54.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj234 td/osd-backfill-space/datafile 2020-09-12T14:45:54.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:54.287 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj234 td/osd-backfill-space/datafile 2020-09-12T14:45:54.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:54.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj234 td/osd-backfill-space/datafile 2020-09-12T14:45:54.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:54.454 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj234 td/osd-backfill-space/datafile 2020-09-12T14:45:54.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:54.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj234 td/osd-backfill-space/datafile 2020-09-12T14:45:54.606 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:54.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj234 td/osd-backfill-space/datafile 2020-09-12T14:45:54.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:54.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj234 td/osd-backfill-space/datafile 2020-09-12T14:45:54.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:54.756 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:54.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:54.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj235 td/osd-backfill-space/datafile 2020-09-12T14:45:54.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:54.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj235 td/osd-backfill-space/datafile 2020-09-12T14:45:54.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:54.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj235 td/osd-backfill-space/datafile 2020-09-12T14:45:55.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:55.007 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj235 td/osd-backfill-space/datafile 2020-09-12T14:45:55.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:55.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj235 td/osd-backfill-space/datafile 2020-09-12T14:45:55.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:55.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj235 td/osd-backfill-space/datafile 2020-09-12T14:45:55.241 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:55.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj235 td/osd-backfill-space/datafile 2020-09-12T14:45:55.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:55.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj235 td/osd-backfill-space/datafile 2020-09-12T14:45:55.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:55.401 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:55.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:55.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj236 td/osd-backfill-space/datafile 2020-09-12T14:45:55.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:55.501 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj236 td/osd-backfill-space/datafile 2020-09-12T14:45:55.576 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:55.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj236 td/osd-backfill-space/datafile 2020-09-12T14:45:55.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:55.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj236 td/osd-backfill-space/datafile 2020-09-12T14:45:55.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:55.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj236 td/osd-backfill-space/datafile 2020-09-12T14:45:55.836 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:55.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj236 td/osd-backfill-space/datafile 2020-09-12T14:45:55.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:55.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj236 td/osd-backfill-space/datafile 2020-09-12T14:45:56.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:56.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj236 td/osd-backfill-space/datafile 2020-09-12T14:45:56.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:56.181 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:56.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:56.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj237 td/osd-backfill-space/datafile 2020-09-12T14:45:56.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:56.255 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj237 td/osd-backfill-space/datafile 2020-09-12T14:45:56.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:56.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj237 td/osd-backfill-space/datafile 2020-09-12T14:45:56.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:56.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj237 td/osd-backfill-space/datafile 2020-09-12T14:45:56.480 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:56.480 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj237 td/osd-backfill-space/datafile 2020-09-12T14:45:56.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:56.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj237 td/osd-backfill-space/datafile 2020-09-12T14:45:56.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:56.631 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj237 td/osd-backfill-space/datafile 2020-09-12T14:45:56.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:56.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj237 td/osd-backfill-space/datafile 2020-09-12T14:45:56.781 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:56.782 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:56.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:56.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj238 td/osd-backfill-space/datafile 2020-09-12T14:45:56.857 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:56.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj238 td/osd-backfill-space/datafile 2020-09-12T14:45:56.932 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:56.933 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj238 td/osd-backfill-space/datafile 2020-09-12T14:45:57.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:57.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj238 td/osd-backfill-space/datafile 2020-09-12T14:45:57.083 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:57.083 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj238 td/osd-backfill-space/datafile 2020-09-12T14:45:57.158 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:57.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj238 td/osd-backfill-space/datafile 2020-09-12T14:45:57.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:57.235 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj238 td/osd-backfill-space/datafile 2020-09-12T14:45:57.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:57.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj238 td/osd-backfill-space/datafile 2020-09-12T14:45:57.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:57.393 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:57.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:57.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj239 td/osd-backfill-space/datafile 2020-09-12T14:45:57.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:57.511 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj239 td/osd-backfill-space/datafile 2020-09-12T14:45:57.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:57.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj239 td/osd-backfill-space/datafile 2020-09-12T14:45:57.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:57.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj239 td/osd-backfill-space/datafile 2020-09-12T14:45:57.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:57.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj239 td/osd-backfill-space/datafile 2020-09-12T14:45:57.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:57.891 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj239 td/osd-backfill-space/datafile 2020-09-12T14:45:57.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:57.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj239 td/osd-backfill-space/datafile 2020-09-12T14:45:58.055 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:58.056 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj239 td/osd-backfill-space/datafile 2020-09-12T14:45:58.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:58.131 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:58.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:58.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj240 td/osd-backfill-space/datafile 2020-09-12T14:45:58.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:58.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj240 td/osd-backfill-space/datafile 2020-09-12T14:45:58.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:58.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj240 td/osd-backfill-space/datafile 2020-09-12T14:45:58.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:58.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj240 td/osd-backfill-space/datafile 2020-09-12T14:45:58.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:58.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj240 td/osd-backfill-space/datafile 2020-09-12T14:45:58.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:58.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj240 td/osd-backfill-space/datafile 2020-09-12T14:45:58.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:58.592 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj240 td/osd-backfill-space/datafile 2020-09-12T14:45:58.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:58.675 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj240 td/osd-backfill-space/datafile 2020-09-12T14:45:58.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:58.760 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:58.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:58.760 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj241 td/osd-backfill-space/datafile 2020-09-12T14:45:58.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:58.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj241 td/osd-backfill-space/datafile 2020-09-12T14:45:58.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:58.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj241 td/osd-backfill-space/datafile 2020-09-12T14:45:58.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:58.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj241 td/osd-backfill-space/datafile 2020-09-12T14:45:59.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:59.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj241 td/osd-backfill-space/datafile 2020-09-12T14:45:59.152 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:59.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj241 td/osd-backfill-space/datafile 2020-09-12T14:45:59.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:59.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj241 td/osd-backfill-space/datafile 2020-09-12T14:45:59.303 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:59.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj241 td/osd-backfill-space/datafile 2020-09-12T14:45:59.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:45:59.379 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:45:59.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:59.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj242 td/osd-backfill-space/datafile 2020-09-12T14:45:59.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:59.471 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj242 td/osd-backfill-space/datafile 2020-09-12T14:45:59.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:59.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj242 td/osd-backfill-space/datafile 2020-09-12T14:45:59.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:59.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj242 td/osd-backfill-space/datafile 2020-09-12T14:45:59.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:59.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj242 td/osd-backfill-space/datafile 2020-09-12T14:45:59.823 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:59.823 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj242 td/osd-backfill-space/datafile 2020-09-12T14:45:59.915 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:59.915 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj242 td/osd-backfill-space/datafile 2020-09-12T14:45:59.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:45:59.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj242 td/osd-backfill-space/datafile 2020-09-12T14:46:00.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:00.075 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:00.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:00.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj243 td/osd-backfill-space/datafile 2020-09-12T14:46:00.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:00.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj243 td/osd-backfill-space/datafile 2020-09-12T14:46:00.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:00.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj243 td/osd-backfill-space/datafile 2020-09-12T14:46:00.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:00.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj243 td/osd-backfill-space/datafile 2020-09-12T14:46:00.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:00.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj243 td/osd-backfill-space/datafile 2020-09-12T14:46:00.468 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:00.468 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj243 td/osd-backfill-space/datafile 2020-09-12T14:46:00.552 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:00.552 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj243 td/osd-backfill-space/datafile 2020-09-12T14:46:00.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:00.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj243 td/osd-backfill-space/datafile 2020-09-12T14:46:00.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:00.703 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:00.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:00.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj244 td/osd-backfill-space/datafile 2020-09-12T14:46:00.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:00.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj244 td/osd-backfill-space/datafile 2020-09-12T14:46:00.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:00.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj244 td/osd-backfill-space/datafile 2020-09-12T14:46:00.928 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:00.929 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj244 td/osd-backfill-space/datafile 2020-09-12T14:46:01.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:01.013 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj244 td/osd-backfill-space/datafile 2020-09-12T14:46:01.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:01.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj244 td/osd-backfill-space/datafile 2020-09-12T14:46:01.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:01.213 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj244 td/osd-backfill-space/datafile 2020-09-12T14:46:01.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:01.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj244 td/osd-backfill-space/datafile 2020-09-12T14:46:01.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:01.364 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:01.365 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:01.365 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj245 td/osd-backfill-space/datafile 2020-09-12T14:46:01.439 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:01.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj245 td/osd-backfill-space/datafile 2020-09-12T14:46:01.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:01.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj245 td/osd-backfill-space/datafile 2020-09-12T14:46:01.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:01.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj245 td/osd-backfill-space/datafile 2020-09-12T14:46:01.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:01.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj245 td/osd-backfill-space/datafile 2020-09-12T14:46:01.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:01.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj245 td/osd-backfill-space/datafile 2020-09-12T14:46:01.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:01.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj245 td/osd-backfill-space/datafile 2020-09-12T14:46:01.953 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:01.953 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj245 td/osd-backfill-space/datafile 2020-09-12T14:46:02.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:02.029 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:02.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:02.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj246 td/osd-backfill-space/datafile 2020-09-12T14:46:02.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:02.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj246 td/osd-backfill-space/datafile 2020-09-12T14:46:02.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:02.188 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj246 td/osd-backfill-space/datafile 2020-09-12T14:46:02.262 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:02.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj246 td/osd-backfill-space/datafile 2020-09-12T14:46:02.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:02.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj246 td/osd-backfill-space/datafile 2020-09-12T14:46:02.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:02.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj246 td/osd-backfill-space/datafile 2020-09-12T14:46:02.481 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:02.481 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj246 td/osd-backfill-space/datafile 2020-09-12T14:46:02.582 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:02.583 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj246 td/osd-backfill-space/datafile 2020-09-12T14:46:02.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:02.657 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:02.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:02.658 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj247 td/osd-backfill-space/datafile 2020-09-12T14:46:02.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:02.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj247 td/osd-backfill-space/datafile 2020-09-12T14:46:02.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:02.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj247 td/osd-backfill-space/datafile 2020-09-12T14:46:02.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:02.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj247 td/osd-backfill-space/datafile 2020-09-12T14:46:02.958 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:02.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj247 td/osd-backfill-space/datafile 2020-09-12T14:46:03.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:03.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj247 td/osd-backfill-space/datafile 2020-09-12T14:46:03.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:03.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj247 td/osd-backfill-space/datafile 2020-09-12T14:46:03.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:03.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj247 td/osd-backfill-space/datafile 2020-09-12T14:46:03.268 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:03.269 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:03.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:03.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj248 td/osd-backfill-space/datafile 2020-09-12T14:46:03.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:03.344 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj248 td/osd-backfill-space/datafile 2020-09-12T14:46:03.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:03.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj248 td/osd-backfill-space/datafile 2020-09-12T14:46:03.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:03.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj248 td/osd-backfill-space/datafile 2020-09-12T14:46:03.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:03.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj248 td/osd-backfill-space/datafile 2020-09-12T14:46:03.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:03.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj248 td/osd-backfill-space/datafile 2020-09-12T14:46:03.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:03.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj248 td/osd-backfill-space/datafile 2020-09-12T14:46:03.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:03.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj248 td/osd-backfill-space/datafile 2020-09-12T14:46:03.997 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:03.998 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:03.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:03.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj249 td/osd-backfill-space/datafile 2020-09-12T14:46:04.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:04.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj249 td/osd-backfill-space/datafile 2020-09-12T14:46:04.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:04.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj249 td/osd-backfill-space/datafile 2020-09-12T14:46:04.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:04.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj249 td/osd-backfill-space/datafile 2020-09-12T14:46:04.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:04.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj249 td/osd-backfill-space/datafile 2020-09-12T14:46:04.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:04.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj249 td/osd-backfill-space/datafile 2020-09-12T14:46:04.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:04.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj249 td/osd-backfill-space/datafile 2020-09-12T14:46:04.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:04.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj249 td/osd-backfill-space/datafile 2020-09-12T14:46:04.600 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:04.600 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:04.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:04.601 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj250 td/osd-backfill-space/datafile 2020-09-12T14:46:04.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:04.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj250 td/osd-backfill-space/datafile 2020-09-12T14:46:04.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:04.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj250 td/osd-backfill-space/datafile 2020-09-12T14:46:04.893 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:04.894 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj250 td/osd-backfill-space/datafile 2020-09-12T14:46:04.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:04.969 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj250 td/osd-backfill-space/datafile 2020-09-12T14:46:05.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:05.044 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj250 td/osd-backfill-space/datafile 2020-09-12T14:46:05.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:05.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj250 td/osd-backfill-space/datafile 2020-09-12T14:46:05.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:05.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj250 td/osd-backfill-space/datafile 2020-09-12T14:46:05.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:05.279 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:05.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:05.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj251 td/osd-backfill-space/datafile 2020-09-12T14:46:05.354 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:05.354 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj251 td/osd-backfill-space/datafile 2020-09-12T14:46:05.438 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:05.438 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj251 td/osd-backfill-space/datafile 2020-09-12T14:46:05.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:05.539 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj251 td/osd-backfill-space/datafile 2020-09-12T14:46:05.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:05.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj251 td/osd-backfill-space/datafile 2020-09-12T14:46:05.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:05.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj251 td/osd-backfill-space/datafile 2020-09-12T14:46:05.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:05.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj251 td/osd-backfill-space/datafile 2020-09-12T14:46:05.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:05.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj251 td/osd-backfill-space/datafile 2020-09-12T14:46:05.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:05.950 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:05.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:05.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj252 td/osd-backfill-space/datafile 2020-09-12T14:46:06.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:06.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj252 td/osd-backfill-space/datafile 2020-09-12T14:46:06.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:06.100 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj252 td/osd-backfill-space/datafile 2020-09-12T14:46:06.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:06.175 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj252 td/osd-backfill-space/datafile 2020-09-12T14:46:06.593 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:06.594 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj252 td/osd-backfill-space/datafile 2020-09-12T14:46:06.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:06.686 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj252 td/osd-backfill-space/datafile 2020-09-12T14:46:06.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:06.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj252 td/osd-backfill-space/datafile 2020-09-12T14:46:06.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:06.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj252 td/osd-backfill-space/datafile 2020-09-12T14:46:06.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:06.945 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:06.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:06.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj253 td/osd-backfill-space/datafile 2020-09-12T14:46:07.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:07.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj253 td/osd-backfill-space/datafile 2020-09-12T14:46:07.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:07.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj253 td/osd-backfill-space/datafile 2020-09-12T14:46:07.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:07.179 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj253 td/osd-backfill-space/datafile 2020-09-12T14:46:07.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:07.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj253 td/osd-backfill-space/datafile 2020-09-12T14:46:07.321 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:07.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj253 td/osd-backfill-space/datafile 2020-09-12T14:46:07.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:07.397 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj253 td/osd-backfill-space/datafile 2020-09-12T14:46:07.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:07.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj253 td/osd-backfill-space/datafile 2020-09-12T14:46:07.578 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:07.579 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:07.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:07.579 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj254 td/osd-backfill-space/datafile 2020-09-12T14:46:07.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:07.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj254 td/osd-backfill-space/datafile 2020-09-12T14:46:07.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:07.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj254 td/osd-backfill-space/datafile 2020-09-12T14:46:07.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:07.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj254 td/osd-backfill-space/datafile 2020-09-12T14:46:07.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:07.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj254 td/osd-backfill-space/datafile 2020-09-12T14:46:08.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:08.026 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj254 td/osd-backfill-space/datafile 2020-09-12T14:46:08.110 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:08.110 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj254 td/osd-backfill-space/datafile 2020-09-12T14:46:08.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:08.185 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj254 td/osd-backfill-space/datafile 2020-09-12T14:46:08.260 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:08.261 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:08.261 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:08.262 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj255 td/osd-backfill-space/datafile 2020-09-12T14:46:08.335 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:08.336 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj255 td/osd-backfill-space/datafile 2020-09-12T14:46:08.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:08.411 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj255 td/osd-backfill-space/datafile 2020-09-12T14:46:08.486 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:08.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj255 td/osd-backfill-space/datafile 2020-09-12T14:46:08.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:08.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj255 td/osd-backfill-space/datafile 2020-09-12T14:46:08.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:08.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj255 td/osd-backfill-space/datafile 2020-09-12T14:46:08.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:08.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj255 td/osd-backfill-space/datafile 2020-09-12T14:46:08.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:08.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj255 td/osd-backfill-space/datafile 2020-09-12T14:46:08.871 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:08.872 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:08.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:08.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj256 td/osd-backfill-space/datafile 2020-09-12T14:46:08.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:08.956 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj256 td/osd-backfill-space/datafile 2020-09-12T14:46:09.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:09.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj256 td/osd-backfill-space/datafile 2020-09-12T14:46:09.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:09.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj256 td/osd-backfill-space/datafile 2020-09-12T14:46:09.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:09.182 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj256 td/osd-backfill-space/datafile 2020-09-12T14:46:09.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:09.257 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj256 td/osd-backfill-space/datafile 2020-09-12T14:46:09.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:09.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj256 td/osd-backfill-space/datafile 2020-09-12T14:46:09.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:09.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj256 td/osd-backfill-space/datafile 2020-09-12T14:46:09.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:09.534 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:09.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:09.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj257 td/osd-backfill-space/datafile 2020-09-12T14:46:09.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:09.609 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj257 td/osd-backfill-space/datafile 2020-09-12T14:46:09.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:09.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj257 td/osd-backfill-space/datafile 2020-09-12T14:46:09.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:09.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj257 td/osd-backfill-space/datafile 2020-09-12T14:46:09.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:09.877 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj257 td/osd-backfill-space/datafile 2020-09-12T14:46:09.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:09.953 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj257 td/osd-backfill-space/datafile 2020-09-12T14:46:10.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:10.028 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj257 td/osd-backfill-space/datafile 2020-09-12T14:46:10.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:10.104 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj257 td/osd-backfill-space/datafile 2020-09-12T14:46:10.178 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:10.179 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:10.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:10.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj258 td/osd-backfill-space/datafile 2020-09-12T14:46:10.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:10.254 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj258 td/osd-backfill-space/datafile 2020-09-12T14:46:10.329 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:10.330 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj258 td/osd-backfill-space/datafile 2020-09-12T14:46:10.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:10.405 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj258 td/osd-backfill-space/datafile 2020-09-12T14:46:10.480 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:10.480 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj258 td/osd-backfill-space/datafile 2020-09-12T14:46:10.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:10.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj258 td/osd-backfill-space/datafile 2020-09-12T14:46:10.622 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:10.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj258 td/osd-backfill-space/datafile 2020-09-12T14:46:10.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:10.706 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj258 td/osd-backfill-space/datafile 2020-09-12T14:46:10.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:10.792 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:10.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:10.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj259 td/osd-backfill-space/datafile 2020-09-12T14:46:10.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:10.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj259 td/osd-backfill-space/datafile 2020-09-12T14:46:10.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:10.950 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj259 td/osd-backfill-space/datafile 2020-09-12T14:46:11.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:11.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj259 td/osd-backfill-space/datafile 2020-09-12T14:46:11.116 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:11.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj259 td/osd-backfill-space/datafile 2020-09-12T14:46:11.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:11.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj259 td/osd-backfill-space/datafile 2020-09-12T14:46:11.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:11.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj259 td/osd-backfill-space/datafile 2020-09-12T14:46:11.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:11.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj259 td/osd-backfill-space/datafile 2020-09-12T14:46:11.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:11.444 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:11.444 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:11.444 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj260 td/osd-backfill-space/datafile 2020-09-12T14:46:11.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:11.544 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj260 td/osd-backfill-space/datafile 2020-09-12T14:46:11.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:11.620 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj260 td/osd-backfill-space/datafile 2020-09-12T14:46:11.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:11.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj260 td/osd-backfill-space/datafile 2020-09-12T14:46:11.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:11.804 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj260 td/osd-backfill-space/datafile 2020-09-12T14:46:12.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:12.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj260 td/osd-backfill-space/datafile 2020-09-12T14:46:12.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:12.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj260 td/osd-backfill-space/datafile 2020-09-12T14:46:12.172 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:12.172 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj260 td/osd-backfill-space/datafile 2020-09-12T14:46:12.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:12.248 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:12.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:12.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj261 td/osd-backfill-space/datafile 2020-09-12T14:46:12.323 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:12.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj261 td/osd-backfill-space/datafile 2020-09-12T14:46:12.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:12.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj261 td/osd-backfill-space/datafile 2020-09-12T14:46:12.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:12.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj261 td/osd-backfill-space/datafile 2020-09-12T14:46:12.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:12.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj261 td/osd-backfill-space/datafile 2020-09-12T14:46:12.691 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:12.692 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj261 td/osd-backfill-space/datafile 2020-09-12T14:46:12.767 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:12.767 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj261 td/osd-backfill-space/datafile 2020-09-12T14:46:12.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:12.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj261 td/osd-backfill-space/datafile 2020-09-12T14:46:12.917 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:12.918 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:12.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:12.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj262 td/osd-backfill-space/datafile 2020-09-12T14:46:12.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:12.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj262 td/osd-backfill-space/datafile 2020-09-12T14:46:13.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:13.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj262 td/osd-backfill-space/datafile 2020-09-12T14:46:13.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:13.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj262 td/osd-backfill-space/datafile 2020-09-12T14:46:13.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:13.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj262 td/osd-backfill-space/datafile 2020-09-12T14:46:13.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:13.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj262 td/osd-backfill-space/datafile 2020-09-12T14:46:13.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:13.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj262 td/osd-backfill-space/datafile 2020-09-12T14:46:13.445 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:13.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj262 td/osd-backfill-space/datafile 2020-09-12T14:46:13.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:13.571 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:13.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:13.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj263 td/osd-backfill-space/datafile 2020-09-12T14:46:13.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:13.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj263 td/osd-backfill-space/datafile 2020-09-12T14:46:13.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:13.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj263 td/osd-backfill-space/datafile 2020-09-12T14:46:13.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:13.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj263 td/osd-backfill-space/datafile 2020-09-12T14:46:13.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:13.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj263 td/osd-backfill-space/datafile 2020-09-12T14:46:13.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:13.982 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj263 td/osd-backfill-space/datafile 2020-09-12T14:46:14.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:14.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj263 td/osd-backfill-space/datafile 2020-09-12T14:46:14.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:14.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj263 td/osd-backfill-space/datafile 2020-09-12T14:46:14.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:14.225 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:14.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:14.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj264 td/osd-backfill-space/datafile 2020-09-12T14:46:14.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:14.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj264 td/osd-backfill-space/datafile 2020-09-12T14:46:14.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:14.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj264 td/osd-backfill-space/datafile 2020-09-12T14:46:14.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:14.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj264 td/osd-backfill-space/datafile 2020-09-12T14:46:14.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:14.527 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj264 td/osd-backfill-space/datafile 2020-09-12T14:46:14.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:14.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj264 td/osd-backfill-space/datafile 2020-09-12T14:46:14.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:14.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj264 td/osd-backfill-space/datafile 2020-09-12T14:46:14.769 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:14.770 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj264 td/osd-backfill-space/datafile 2020-09-12T14:46:14.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:14.845 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:14.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:14.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj265 td/osd-backfill-space/datafile 2020-09-12T14:46:14.928 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:14.928 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj265 td/osd-backfill-space/datafile 2020-09-12T14:46:15.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:15.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj265 td/osd-backfill-space/datafile 2020-09-12T14:46:15.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:15.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj265 td/osd-backfill-space/datafile 2020-09-12T14:46:15.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:15.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj265 td/osd-backfill-space/datafile 2020-09-12T14:46:15.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:15.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj265 td/osd-backfill-space/datafile 2020-09-12T14:46:15.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:15.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj265 td/osd-backfill-space/datafile 2020-09-12T14:46:15.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:15.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj265 td/osd-backfill-space/datafile 2020-09-12T14:46:15.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:15.449 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:15.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:15.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj266 td/osd-backfill-space/datafile 2020-09-12T14:46:15.573 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:15.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj266 td/osd-backfill-space/datafile 2020-09-12T14:46:15.649 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:15.649 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj266 td/osd-backfill-space/datafile 2020-09-12T14:46:15.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:15.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj266 td/osd-backfill-space/datafile 2020-09-12T14:46:15.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:15.825 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj266 td/osd-backfill-space/datafile 2020-09-12T14:46:15.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:15.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj266 td/osd-backfill-space/datafile 2020-09-12T14:46:15.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:15.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj266 td/osd-backfill-space/datafile 2020-09-12T14:46:16.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:16.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj266 td/osd-backfill-space/datafile 2020-09-12T14:46:16.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:16.144 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:16.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:16.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj267 td/osd-backfill-space/datafile 2020-09-12T14:46:16.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:16.219 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj267 td/osd-backfill-space/datafile 2020-09-12T14:46:16.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:16.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj267 td/osd-backfill-space/datafile 2020-09-12T14:46:16.386 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:16.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj267 td/osd-backfill-space/datafile 2020-09-12T14:46:16.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:16.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj267 td/osd-backfill-space/datafile 2020-09-12T14:46:16.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:16.537 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj267 td/osd-backfill-space/datafile 2020-09-12T14:46:16.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:16.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj267 td/osd-backfill-space/datafile 2020-09-12T14:46:16.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:16.697 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj267 td/osd-backfill-space/datafile 2020-09-12T14:46:16.772 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:16.772 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:16.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:16.773 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj268 td/osd-backfill-space/datafile 2020-09-12T14:46:16.855 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:16.856 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj268 td/osd-backfill-space/datafile 2020-09-12T14:46:16.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:16.932 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj268 td/osd-backfill-space/datafile 2020-09-12T14:46:17.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:17.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj268 td/osd-backfill-space/datafile 2020-09-12T14:46:17.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:17.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj268 td/osd-backfill-space/datafile 2020-09-12T14:46:17.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:17.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj268 td/osd-backfill-space/datafile 2020-09-12T14:46:17.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:17.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj268 td/osd-backfill-space/datafile 2020-09-12T14:46:17.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:17.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj268 td/osd-backfill-space/datafile 2020-09-12T14:46:17.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:17.462 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:17.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:17.463 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj269 td/osd-backfill-space/datafile 2020-09-12T14:46:17.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:17.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj269 td/osd-backfill-space/datafile 2020-09-12T14:46:17.626 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:17.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj269 td/osd-backfill-space/datafile 2020-09-12T14:46:17.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:17.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj269 td/osd-backfill-space/datafile 2020-09-12T14:46:17.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:17.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj269 td/osd-backfill-space/datafile 2020-09-12T14:46:17.936 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:17.937 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj269 td/osd-backfill-space/datafile 2020-09-12T14:46:18.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:18.012 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj269 td/osd-backfill-space/datafile 2020-09-12T14:46:18.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:18.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj269 td/osd-backfill-space/datafile 2020-09-12T14:46:18.163 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:18.163 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:18.164 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:18.164 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj270 td/osd-backfill-space/datafile 2020-09-12T14:46:18.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:18.238 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj270 td/osd-backfill-space/datafile 2020-09-12T14:46:18.313 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:18.314 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj270 td/osd-backfill-space/datafile 2020-09-12T14:46:18.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:18.401 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj270 td/osd-backfill-space/datafile 2020-09-12T14:46:18.481 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:18.481 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj270 td/osd-backfill-space/datafile 2020-09-12T14:46:18.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:18.557 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj270 td/osd-backfill-space/datafile 2020-09-12T14:46:18.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:18.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj270 td/osd-backfill-space/datafile 2020-09-12T14:46:18.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:18.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj270 td/osd-backfill-space/datafile 2020-09-12T14:46:18.782 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:18.783 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:18.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:18.783 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj271 td/osd-backfill-space/datafile 2020-09-12T14:46:18.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:18.858 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj271 td/osd-backfill-space/datafile 2020-09-12T14:46:18.933 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:18.934 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj271 td/osd-backfill-space/datafile 2020-09-12T14:46:19.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:19.009 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj271 td/osd-backfill-space/datafile 2020-09-12T14:46:19.084 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:19.084 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj271 td/osd-backfill-space/datafile 2020-09-12T14:46:19.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:19.168 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj271 td/osd-backfill-space/datafile 2020-09-12T14:46:19.243 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:19.244 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj271 td/osd-backfill-space/datafile 2020-09-12T14:46:19.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:19.320 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj271 td/osd-backfill-space/datafile 2020-09-12T14:46:19.394 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:19.394 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:19.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:19.395 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj272 td/osd-backfill-space/datafile 2020-09-12T14:46:19.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:19.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj272 td/osd-backfill-space/datafile 2020-09-12T14:46:19.570 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:19.571 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj272 td/osd-backfill-space/datafile 2020-09-12T14:46:19.645 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:19.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj272 td/osd-backfill-space/datafile 2020-09-12T14:46:19.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:19.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj272 td/osd-backfill-space/datafile 2020-09-12T14:46:19.846 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:19.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj272 td/osd-backfill-space/datafile 2020-09-12T14:46:19.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:19.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj272 td/osd-backfill-space/datafile 2020-09-12T14:46:20.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:20.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj272 td/osd-backfill-space/datafile 2020-09-12T14:46:20.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:20.082 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:20.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:20.082 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj273 td/osd-backfill-space/datafile 2020-09-12T14:46:20.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:20.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj273 td/osd-backfill-space/datafile 2020-09-12T14:46:20.250 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:20.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj273 td/osd-backfill-space/datafile 2020-09-12T14:46:20.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:20.326 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj273 td/osd-backfill-space/datafile 2020-09-12T14:46:20.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:20.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj273 td/osd-backfill-space/datafile 2020-09-12T14:46:20.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:20.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj273 td/osd-backfill-space/datafile 2020-09-12T14:46:20.550 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:20.551 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj273 td/osd-backfill-space/datafile 2020-09-12T14:46:20.626 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:20.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj273 td/osd-backfill-space/datafile 2020-09-12T14:46:20.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:20.702 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:20.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:20.704 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj274 td/osd-backfill-space/datafile 2020-09-12T14:46:20.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:20.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj274 td/osd-backfill-space/datafile 2020-09-12T14:46:20.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:20.853 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj274 td/osd-backfill-space/datafile 2020-09-12T14:46:20.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:20.928 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj274 td/osd-backfill-space/datafile 2020-09-12T14:46:21.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:21.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj274 td/osd-backfill-space/datafile 2020-09-12T14:46:21.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:21.079 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj274 td/osd-backfill-space/datafile 2020-09-12T14:46:21.154 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:21.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj274 td/osd-backfill-space/datafile 2020-09-12T14:46:21.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:21.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj274 td/osd-backfill-space/datafile 2020-09-12T14:46:21.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:21.304 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:21.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:21.305 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj275 td/osd-backfill-space/datafile 2020-09-12T14:46:21.389 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:21.390 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj275 td/osd-backfill-space/datafile 2020-09-12T14:46:21.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:21.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj275 td/osd-backfill-space/datafile 2020-09-12T14:46:21.573 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:21.573 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj275 td/osd-backfill-space/datafile 2020-09-12T14:46:21.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:21.648 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj275 td/osd-backfill-space/datafile 2020-09-12T14:46:21.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:21.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj275 td/osd-backfill-space/datafile 2020-09-12T14:46:21.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:21.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj275 td/osd-backfill-space/datafile 2020-09-12T14:46:21.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:21.900 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj275 td/osd-backfill-space/datafile 2020-09-12T14:46:21.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:21.994 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:21.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:21.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj276 td/osd-backfill-space/datafile 2020-09-12T14:46:22.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:22.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj276 td/osd-backfill-space/datafile 2020-09-12T14:46:22.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:22.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj276 td/osd-backfill-space/datafile 2020-09-12T14:46:22.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:22.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj276 td/osd-backfill-space/datafile 2020-09-12T14:46:22.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:22.294 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj276 td/osd-backfill-space/datafile 2020-09-12T14:46:22.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:22.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj276 td/osd-backfill-space/datafile 2020-09-12T14:46:22.444 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:22.444 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj276 td/osd-backfill-space/datafile 2020-09-12T14:46:22.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:22.528 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj276 td/osd-backfill-space/datafile 2020-09-12T14:46:22.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:22.603 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:22.604 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:22.604 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj277 td/osd-backfill-space/datafile 2020-09-12T14:46:22.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:22.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj277 td/osd-backfill-space/datafile 2020-09-12T14:46:22.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:22.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj277 td/osd-backfill-space/datafile 2020-09-12T14:46:22.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:22.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj277 td/osd-backfill-space/datafile 2020-09-12T14:46:22.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:22.972 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj277 td/osd-backfill-space/datafile 2020-09-12T14:46:23.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:23.047 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj277 td/osd-backfill-space/datafile 2020-09-12T14:46:23.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:23.133 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj277 td/osd-backfill-space/datafile 2020-09-12T14:46:23.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:23.206 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj277 td/osd-backfill-space/datafile 2020-09-12T14:46:23.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:23.282 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:23.282 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:23.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj278 td/osd-backfill-space/datafile 2020-09-12T14:46:23.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:23.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj278 td/osd-backfill-space/datafile 2020-09-12T14:46:23.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:23.433 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj278 td/osd-backfill-space/datafile 2020-09-12T14:46:23.550 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:23.550 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj278 td/osd-backfill-space/datafile 2020-09-12T14:46:23.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:23.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj278 td/osd-backfill-space/datafile 2020-09-12T14:46:23.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:23.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj278 td/osd-backfill-space/datafile 2020-09-12T14:46:23.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:23.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj278 td/osd-backfill-space/datafile 2020-09-12T14:46:23.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:23.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj278 td/osd-backfill-space/datafile 2020-09-12T14:46:24.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:24.003 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:24.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:24.003 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj279 td/osd-backfill-space/datafile 2020-09-12T14:46:24.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:24.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj279 td/osd-backfill-space/datafile 2020-09-12T14:46:24.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:24.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj279 td/osd-backfill-space/datafile 2020-09-12T14:46:24.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:24.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj279 td/osd-backfill-space/datafile 2020-09-12T14:46:24.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:24.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj279 td/osd-backfill-space/datafile 2020-09-12T14:46:24.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:24.380 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj279 td/osd-backfill-space/datafile 2020-09-12T14:46:24.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:24.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj279 td/osd-backfill-space/datafile 2020-09-12T14:46:24.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:24.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj279 td/osd-backfill-space/datafile 2020-09-12T14:46:24.606 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:24.606 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:24.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:24.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj280 td/osd-backfill-space/datafile 2020-09-12T14:46:24.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:24.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj280 td/osd-backfill-space/datafile 2020-09-12T14:46:24.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:24.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj280 td/osd-backfill-space/datafile 2020-09-12T14:46:24.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:24.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj280 td/osd-backfill-space/datafile 2020-09-12T14:46:24.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:24.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj280 td/osd-backfill-space/datafile 2020-09-12T14:46:24.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:24.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj280 td/osd-backfill-space/datafile 2020-09-12T14:46:25.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:25.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj280 td/osd-backfill-space/datafile 2020-09-12T14:46:25.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:25.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj280 td/osd-backfill-space/datafile 2020-09-12T14:46:25.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:25.226 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:25.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:25.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj281 td/osd-backfill-space/datafile 2020-09-12T14:46:25.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:25.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj281 td/osd-backfill-space/datafile 2020-09-12T14:46:25.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:25.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj281 td/osd-backfill-space/datafile 2020-09-12T14:46:25.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:25.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj281 td/osd-backfill-space/datafile 2020-09-12T14:46:25.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:25.553 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj281 td/osd-backfill-space/datafile 2020-09-12T14:46:25.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:25.628 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj281 td/osd-backfill-space/datafile 2020-09-12T14:46:25.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:25.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj281 td/osd-backfill-space/datafile 2020-09-12T14:46:25.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:25.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj281 td/osd-backfill-space/datafile 2020-09-12T14:46:25.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:25.913 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:25.914 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:25.914 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj282 td/osd-backfill-space/datafile 2020-09-12T14:46:25.996 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:25.997 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj282 td/osd-backfill-space/datafile 2020-09-12T14:46:26.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:26.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj282 td/osd-backfill-space/datafile 2020-09-12T14:46:26.147 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:26.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj282 td/osd-backfill-space/datafile 2020-09-12T14:46:26.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:26.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj282 td/osd-backfill-space/datafile 2020-09-12T14:46:26.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:26.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj282 td/osd-backfill-space/datafile 2020-09-12T14:46:26.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:26.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj282 td/osd-backfill-space/datafile 2020-09-12T14:46:26.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:26.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj282 td/osd-backfill-space/datafile 2020-09-12T14:46:26.524 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:26.525 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:26.525 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:26.526 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj283 td/osd-backfill-space/datafile 2020-09-12T14:46:26.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:26.608 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj283 td/osd-backfill-space/datafile 2020-09-12T14:46:26.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:26.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj283 td/osd-backfill-space/datafile 2020-09-12T14:46:26.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:26.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj283 td/osd-backfill-space/datafile 2020-09-12T14:46:26.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:26.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj283 td/osd-backfill-space/datafile 2020-09-12T14:46:26.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:26.918 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj283 td/osd-backfill-space/datafile 2020-09-12T14:46:26.993 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:26.994 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj283 td/osd-backfill-space/datafile 2020-09-12T14:46:27.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:27.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj283 td/osd-backfill-space/datafile 2020-09-12T14:46:27.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:27.145 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:27.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:27.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj284 td/osd-backfill-space/datafile 2020-09-12T14:46:27.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:27.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj284 td/osd-backfill-space/datafile 2020-09-12T14:46:27.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:27.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj284 td/osd-backfill-space/datafile 2020-09-12T14:46:27.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:27.555 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj284 td/osd-backfill-space/datafile 2020-09-12T14:46:27.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:27.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj284 td/osd-backfill-space/datafile 2020-09-12T14:46:27.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:27.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj284 td/osd-backfill-space/datafile 2020-09-12T14:46:27.839 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:27.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj284 td/osd-backfill-space/datafile 2020-09-12T14:46:27.915 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:27.916 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj284 td/osd-backfill-space/datafile 2020-09-12T14:46:27.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:27.999 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:27.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:28.000 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj285 td/osd-backfill-space/datafile 2020-09-12T14:46:28.066 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:28.066 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj285 td/osd-backfill-space/datafile 2020-09-12T14:46:28.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:28.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj285 td/osd-backfill-space/datafile 2020-09-12T14:46:28.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:28.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj285 td/osd-backfill-space/datafile 2020-09-12T14:46:28.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:28.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj285 td/osd-backfill-space/datafile 2020-09-12T14:46:28.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:28.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj285 td/osd-backfill-space/datafile 2020-09-12T14:46:28.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:28.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj285 td/osd-backfill-space/datafile 2020-09-12T14:46:28.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:28.535 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj285 td/osd-backfill-space/datafile 2020-09-12T14:46:28.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:28.611 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:28.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:28.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj286 td/osd-backfill-space/datafile 2020-09-12T14:46:28.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:28.695 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj286 td/osd-backfill-space/datafile 2020-09-12T14:46:28.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:28.778 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj286 td/osd-backfill-space/datafile 2020-09-12T14:46:28.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:28.854 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj286 td/osd-backfill-space/datafile 2020-09-12T14:46:28.929 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:28.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj286 td/osd-backfill-space/datafile 2020-09-12T14:46:29.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:29.004 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj286 td/osd-backfill-space/datafile 2020-09-12T14:46:29.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:29.080 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj286 td/osd-backfill-space/datafile 2020-09-12T14:46:29.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:29.155 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj286 td/osd-backfill-space/datafile 2020-09-12T14:46:29.230 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:29.231 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:29.231 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:29.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj287 td/osd-backfill-space/datafile 2020-09-12T14:46:29.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:29.306 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj287 td/osd-backfill-space/datafile 2020-09-12T14:46:29.381 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:29.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj287 td/osd-backfill-space/datafile 2020-09-12T14:46:29.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:29.462 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj287 td/osd-backfill-space/datafile 2020-09-12T14:46:29.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:29.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj287 td/osd-backfill-space/datafile 2020-09-12T14:46:29.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:29.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj287 td/osd-backfill-space/datafile 2020-09-12T14:46:29.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:29.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj287 td/osd-backfill-space/datafile 2020-09-12T14:46:29.817 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:29.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj287 td/osd-backfill-space/datafile 2020-09-12T14:46:29.892 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:29.893 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:29.894 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:29.894 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj288 td/osd-backfill-space/datafile 2020-09-12T14:46:29.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:29.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj288 td/osd-backfill-space/datafile 2020-09-12T14:46:30.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:30.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj288 td/osd-backfill-space/datafile 2020-09-12T14:46:30.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:30.128 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj288 td/osd-backfill-space/datafile 2020-09-12T14:46:30.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:30.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj288 td/osd-backfill-space/datafile 2020-09-12T14:46:30.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:30.279 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj288 td/osd-backfill-space/datafile 2020-09-12T14:46:30.353 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:30.354 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj288 td/osd-backfill-space/datafile 2020-09-12T14:46:30.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:30.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj288 td/osd-backfill-space/datafile 2020-09-12T14:46:30.496 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:30.497 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:30.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:30.497 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj289 td/osd-backfill-space/datafile 2020-09-12T14:46:30.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:30.580 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj289 td/osd-backfill-space/datafile 2020-09-12T14:46:30.655 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:30.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj289 td/osd-backfill-space/datafile 2020-09-12T14:46:30.730 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:30.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj289 td/osd-backfill-space/datafile 2020-09-12T14:46:30.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:30.806 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj289 td/osd-backfill-space/datafile 2020-09-12T14:46:30.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:30.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj289 td/osd-backfill-space/datafile 2020-09-12T14:46:30.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:30.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj289 td/osd-backfill-space/datafile 2020-09-12T14:46:31.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:31.041 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj289 td/osd-backfill-space/datafile 2020-09-12T14:46:31.116 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:31.117 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:31.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:31.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj290 td/osd-backfill-space/datafile 2020-09-12T14:46:31.191 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:31.191 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj290 td/osd-backfill-space/datafile 2020-09-12T14:46:31.266 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:31.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj290 td/osd-backfill-space/datafile 2020-09-12T14:46:31.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:31.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj290 td/osd-backfill-space/datafile 2020-09-12T14:46:31.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:31.426 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj290 td/osd-backfill-space/datafile 2020-09-12T14:46:31.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:31.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj290 td/osd-backfill-space/datafile 2020-09-12T14:46:31.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:31.619 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj290 td/osd-backfill-space/datafile 2020-09-12T14:46:31.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:31.703 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj290 td/osd-backfill-space/datafile 2020-09-12T14:46:31.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:31.829 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:31.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:31.830 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj291 td/osd-backfill-space/datafile 2020-09-12T14:46:31.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:31.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj291 td/osd-backfill-space/datafile 2020-09-12T14:46:31.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:31.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj291 td/osd-backfill-space/datafile 2020-09-12T14:46:32.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:32.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj291 td/osd-backfill-space/datafile 2020-09-12T14:46:32.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:32.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj291 td/osd-backfill-space/datafile 2020-09-12T14:46:32.222 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:32.222 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj291 td/osd-backfill-space/datafile 2020-09-12T14:46:32.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:32.298 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj291 td/osd-backfill-space/datafile 2020-09-12T14:46:32.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:32.398 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj291 td/osd-backfill-space/datafile 2020-09-12T14:46:32.473 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:32.474 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:32.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:32.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj292 td/osd-backfill-space/datafile 2020-09-12T14:46:32.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:32.549 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj292 td/osd-backfill-space/datafile 2020-09-12T14:46:32.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:32.624 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj292 td/osd-backfill-space/datafile 2020-09-12T14:46:32.699 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:32.700 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj292 td/osd-backfill-space/datafile 2020-09-12T14:46:32.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:32.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj292 td/osd-backfill-space/datafile 2020-09-12T14:46:32.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:32.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj292 td/osd-backfill-space/datafile 2020-09-12T14:46:32.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:32.960 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj292 td/osd-backfill-space/datafile 2020-09-12T14:46:33.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:33.035 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj292 td/osd-backfill-space/datafile 2020-09-12T14:46:33.110 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:33.110 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:33.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:33.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj293 td/osd-backfill-space/datafile 2020-09-12T14:46:33.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:33.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj293 td/osd-backfill-space/datafile 2020-09-12T14:46:33.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:33.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj293 td/osd-backfill-space/datafile 2020-09-12T14:46:33.369 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:33.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj293 td/osd-backfill-space/datafile 2020-09-12T14:46:33.445 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:33.445 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj293 td/osd-backfill-space/datafile 2020-09-12T14:46:33.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:33.554 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj293 td/osd-backfill-space/datafile 2020-09-12T14:46:33.629 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:33.630 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj293 td/osd-backfill-space/datafile 2020-09-12T14:46:33.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:33.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj293 td/osd-backfill-space/datafile 2020-09-12T14:46:33.831 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:33.831 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:33.832 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:33.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj294 td/osd-backfill-space/datafile 2020-09-12T14:46:33.906 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:33.907 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj294 td/osd-backfill-space/datafile 2020-09-12T14:46:33.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:33.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj294 td/osd-backfill-space/datafile 2020-09-12T14:46:34.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:34.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj294 td/osd-backfill-space/datafile 2020-09-12T14:46:34.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:34.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj294 td/osd-backfill-space/datafile 2020-09-12T14:46:34.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:34.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj294 td/osd-backfill-space/datafile 2020-09-12T14:46:34.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:34.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj294 td/osd-backfill-space/datafile 2020-09-12T14:46:34.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:34.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj294 td/osd-backfill-space/datafile 2020-09-12T14:46:34.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:34.459 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:34.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:34.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj295 td/osd-backfill-space/datafile 2020-09-12T14:46:34.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:34.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj295 td/osd-backfill-space/datafile 2020-09-12T14:46:34.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:34.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj295 td/osd-backfill-space/datafile 2020-09-12T14:46:34.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:34.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj295 td/osd-backfill-space/datafile 2020-09-12T14:46:34.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:34.786 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj295 td/osd-backfill-space/datafile 2020-09-12T14:46:34.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:34.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj295 td/osd-backfill-space/datafile 2020-09-12T14:46:34.953 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:34.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj295 td/osd-backfill-space/datafile 2020-09-12T14:46:35.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:35.029 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj295 td/osd-backfill-space/datafile 2020-09-12T14:46:35.121 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:35.121 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:35.122 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:35.122 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj296 td/osd-backfill-space/datafile 2020-09-12T14:46:35.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:35.204 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj296 td/osd-backfill-space/datafile 2020-09-12T14:46:35.280 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:35.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj296 td/osd-backfill-space/datafile 2020-09-12T14:46:35.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:35.364 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj296 td/osd-backfill-space/datafile 2020-09-12T14:46:35.447 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:35.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj296 td/osd-backfill-space/datafile 2020-09-12T14:46:35.556 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:35.557 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj296 td/osd-backfill-space/datafile 2020-09-12T14:46:35.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:35.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj296 td/osd-backfill-space/datafile 2020-09-12T14:46:35.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:35.707 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj296 td/osd-backfill-space/datafile 2020-09-12T14:46:35.850 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:35.850 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:35.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:35.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj297 td/osd-backfill-space/datafile 2020-09-12T14:46:35.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:35.925 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj297 td/osd-backfill-space/datafile 2020-09-12T14:46:36.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:36.009 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj297 td/osd-backfill-space/datafile 2020-09-12T14:46:36.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:36.101 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj297 td/osd-backfill-space/datafile 2020-09-12T14:46:36.176 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:36.177 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj297 td/osd-backfill-space/datafile 2020-09-12T14:46:36.251 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:36.252 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj297 td/osd-backfill-space/datafile 2020-09-12T14:46:36.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:36.327 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj297 td/osd-backfill-space/datafile 2020-09-12T14:46:36.402 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:36.403 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj297 td/osd-backfill-space/datafile 2020-09-12T14:46:36.478 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:36.478 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:36.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:36.479 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj298 td/osd-backfill-space/datafile 2020-09-12T14:46:36.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:36.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj298 td/osd-backfill-space/datafile 2020-09-12T14:46:36.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:36.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj298 td/osd-backfill-space/datafile 2020-09-12T14:46:36.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:36.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj298 td/osd-backfill-space/datafile 2020-09-12T14:46:36.796 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:36.797 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj298 td/osd-backfill-space/datafile 2020-09-12T14:46:36.871 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:36.872 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj298 td/osd-backfill-space/datafile 2020-09-12T14:46:36.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:36.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj298 td/osd-backfill-space/datafile 2020-09-12T14:46:37.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:37.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj298 td/osd-backfill-space/datafile 2020-09-12T14:46:37.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:37.098 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:37.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:37.099 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj299 td/osd-backfill-space/datafile 2020-09-12T14:46:37.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:37.173 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj299 td/osd-backfill-space/datafile 2020-09-12T14:46:37.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:37.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj299 td/osd-backfill-space/datafile 2020-09-12T14:46:37.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:37.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj299 td/osd-backfill-space/datafile 2020-09-12T14:46:37.399 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:37.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj299 td/osd-backfill-space/datafile 2020-09-12T14:46:37.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:37.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj299 td/osd-backfill-space/datafile 2020-09-12T14:46:37.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:37.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj299 td/osd-backfill-space/datafile 2020-09-12T14:46:38.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:38.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj299 td/osd-backfill-space/datafile 2020-09-12T14:46:38.169 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:38.171 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:38.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:38.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj300 td/osd-backfill-space/datafile 2020-09-12T14:46:38.261 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:38.261 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj300 td/osd-backfill-space/datafile 2020-09-12T14:46:38.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:38.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj300 td/osd-backfill-space/datafile 2020-09-12T14:46:38.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:38.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj300 td/osd-backfill-space/datafile 2020-09-12T14:46:38.512 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:38.513 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj300 td/osd-backfill-space/datafile 2020-09-12T14:46:38.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:38.588 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj300 td/osd-backfill-space/datafile 2020-09-12T14:46:38.663 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:38.663 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj300 td/osd-backfill-space/datafile 2020-09-12T14:46:38.738 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:38.739 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj300 td/osd-backfill-space/datafile 2020-09-12T14:46:38.814 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:38.814 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:38.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:38.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj301 td/osd-backfill-space/datafile 2020-09-12T14:46:38.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:38.889 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj301 td/osd-backfill-space/datafile 2020-09-12T14:46:38.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:38.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj301 td/osd-backfill-space/datafile 2020-09-12T14:46:39.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:39.040 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj301 td/osd-backfill-space/datafile 2020-09-12T14:46:39.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:39.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj301 td/osd-backfill-space/datafile 2020-09-12T14:46:39.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:39.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj301 td/osd-backfill-space/datafile 2020-09-12T14:46:39.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:39.275 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj301 td/osd-backfill-space/datafile 2020-09-12T14:46:39.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:39.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj301 td/osd-backfill-space/datafile 2020-09-12T14:46:39.434 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:39.434 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:39.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:39.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj302 td/osd-backfill-space/datafile 2020-09-12T14:46:39.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:39.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj302 td/osd-backfill-space/datafile 2020-09-12T14:46:39.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:39.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj302 td/osd-backfill-space/datafile 2020-09-12T14:46:39.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:39.694 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj302 td/osd-backfill-space/datafile 2020-09-12T14:46:39.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:39.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj302 td/osd-backfill-space/datafile 2020-09-12T14:46:39.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:39.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj302 td/osd-backfill-space/datafile 2020-09-12T14:46:39.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:39.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj302 td/osd-backfill-space/datafile 2020-09-12T14:46:40.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:40.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj302 td/osd-backfill-space/datafile 2020-09-12T14:46:40.096 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:40.096 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:40.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:40.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj303 td/osd-backfill-space/datafile 2020-09-12T14:46:40.171 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:40.172 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj303 td/osd-backfill-space/datafile 2020-09-12T14:46:40.246 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:40.247 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj303 td/osd-backfill-space/datafile 2020-09-12T14:46:40.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:40.322 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj303 td/osd-backfill-space/datafile 2020-09-12T14:46:40.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:40.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj303 td/osd-backfill-space/datafile 2020-09-12T14:46:40.481 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:40.482 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj303 td/osd-backfill-space/datafile 2020-09-12T14:46:40.557 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:40.557 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj303 td/osd-backfill-space/datafile 2020-09-12T14:46:40.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:40.632 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj303 td/osd-backfill-space/datafile 2020-09-12T14:46:40.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:40.720 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:40.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:40.721 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj304 td/osd-backfill-space/datafile 2020-09-12T14:46:40.799 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:40.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj304 td/osd-backfill-space/datafile 2020-09-12T14:46:40.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:40.884 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj304 td/osd-backfill-space/datafile 2020-09-12T14:46:40.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:40.959 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj304 td/osd-backfill-space/datafile 2020-09-12T14:46:41.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:41.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj304 td/osd-backfill-space/datafile 2020-09-12T14:46:41.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:41.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj304 td/osd-backfill-space/datafile 2020-09-12T14:46:41.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:41.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj304 td/osd-backfill-space/datafile 2020-09-12T14:46:41.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:41.286 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj304 td/osd-backfill-space/datafile 2020-09-12T14:46:41.361 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:41.361 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:41.362 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:41.363 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj305 td/osd-backfill-space/datafile 2020-09-12T14:46:41.437 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:41.437 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj305 td/osd-backfill-space/datafile 2020-09-12T14:46:41.545 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:41.546 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj305 td/osd-backfill-space/datafile 2020-09-12T14:46:41.621 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:41.621 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj305 td/osd-backfill-space/datafile 2020-09-12T14:46:41.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:41.696 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj305 td/osd-backfill-space/datafile 2020-09-12T14:46:41.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:41.805 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj305 td/osd-backfill-space/datafile 2020-09-12T14:46:41.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:41.880 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj305 td/osd-backfill-space/datafile 2020-09-12T14:46:41.947 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:41.948 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj305 td/osd-backfill-space/datafile 2020-09-12T14:46:42.023 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:42.023 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:42.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:42.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj306 td/osd-backfill-space/datafile 2020-09-12T14:46:42.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:42.098 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj306 td/osd-backfill-space/datafile 2020-09-12T14:46:42.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:42.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj306 td/osd-backfill-space/datafile 2020-09-12T14:46:42.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:42.249 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj306 td/osd-backfill-space/datafile 2020-09-12T14:46:42.324 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:42.325 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj306 td/osd-backfill-space/datafile 2020-09-12T14:46:42.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:42.400 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj306 td/osd-backfill-space/datafile 2020-09-12T14:46:42.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:42.492 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj306 td/osd-backfill-space/datafile 2020-09-12T14:46:42.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:42.568 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj306 td/osd-backfill-space/datafile 2020-09-12T14:46:42.651 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:42.651 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:42.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:42.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj307 td/osd-backfill-space/datafile 2020-09-12T14:46:42.726 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:42.727 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj307 td/osd-backfill-space/datafile 2020-09-12T14:46:42.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:42.802 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj307 td/osd-backfill-space/datafile 2020-09-12T14:46:42.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:42.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj307 td/osd-backfill-space/datafile 2020-09-12T14:46:43.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:43.005 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj307 td/osd-backfill-space/datafile 2020-09-12T14:46:43.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:43.162 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj307 td/osd-backfill-space/datafile 2020-09-12T14:46:43.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:43.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj307 td/osd-backfill-space/datafile 2020-09-12T14:46:43.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:43.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj307 td/osd-backfill-space/datafile 2020-09-12T14:46:43.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:43.422 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:43.422 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:43.422 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj308 td/osd-backfill-space/datafile 2020-09-12T14:46:43.547 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:43.547 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj308 td/osd-backfill-space/datafile 2020-09-12T14:46:43.622 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:43.623 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj308 td/osd-backfill-space/datafile 2020-09-12T14:46:43.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:43.698 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj308 td/osd-backfill-space/datafile 2020-09-12T14:46:43.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:43.815 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj308 td/osd-backfill-space/datafile 2020-09-12T14:46:43.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:43.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj308 td/osd-backfill-space/datafile 2020-09-12T14:46:43.974 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:43.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj308 td/osd-backfill-space/datafile 2020-09-12T14:46:44.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:44.050 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj308 td/osd-backfill-space/datafile 2020-09-12T14:46:44.125 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:44.126 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:44.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:44.126 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj309 td/osd-backfill-space/datafile 2020-09-12T14:46:44.209 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:44.210 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj309 td/osd-backfill-space/datafile 2020-09-12T14:46:44.284 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:44.285 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj309 td/osd-backfill-space/datafile 2020-09-12T14:46:44.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:44.360 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj309 td/osd-backfill-space/datafile 2020-09-12T14:46:44.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:44.435 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj309 td/osd-backfill-space/datafile 2020-09-12T14:46:44.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:44.519 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj309 td/osd-backfill-space/datafile 2020-09-12T14:46:44.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:44.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj309 td/osd-backfill-space/datafile 2020-09-12T14:46:44.678 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:44.679 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj309 td/osd-backfill-space/datafile 2020-09-12T14:46:44.754 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:44.754 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:44.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:44.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj310 td/osd-backfill-space/datafile 2020-09-12T14:46:44.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:44.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj310 td/osd-backfill-space/datafile 2020-09-12T14:46:44.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:44.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj310 td/osd-backfill-space/datafile 2020-09-12T14:46:44.990 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:44.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj310 td/osd-backfill-space/datafile 2020-09-12T14:46:45.063 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:45.064 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj310 td/osd-backfill-space/datafile 2020-09-12T14:46:45.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:45.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj310 td/osd-backfill-space/datafile 2020-09-12T14:46:45.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:45.215 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj310 td/osd-backfill-space/datafile 2020-09-12T14:46:45.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:45.290 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj310 td/osd-backfill-space/datafile 2020-09-12T14:46:45.365 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:45.365 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:45.366 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:45.366 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj311 td/osd-backfill-space/datafile 2020-09-12T14:46:45.457 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:45.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj311 td/osd-backfill-space/datafile 2020-09-12T14:46:45.566 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:45.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj311 td/osd-backfill-space/datafile 2020-09-12T14:46:45.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:45.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj311 td/osd-backfill-space/datafile 2020-09-12T14:46:45.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:45.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj311 td/osd-backfill-space/datafile 2020-09-12T14:46:45.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:45.835 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj311 td/osd-backfill-space/datafile 2020-09-12T14:46:45.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:45.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj311 td/osd-backfill-space/datafile 2020-09-12T14:46:45.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:45.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj311 td/osd-backfill-space/datafile 2020-09-12T14:46:46.069 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:46.070 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:46.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:46.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj312 td/osd-backfill-space/datafile 2020-09-12T14:46:46.144 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:46.145 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj312 td/osd-backfill-space/datafile 2020-09-12T14:46:46.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:46.220 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj312 td/osd-backfill-space/datafile 2020-09-12T14:46:46.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:46.304 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj312 td/osd-backfill-space/datafile 2020-09-12T14:46:46.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:46.379 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj312 td/osd-backfill-space/datafile 2020-09-12T14:46:46.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:46.455 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj312 td/osd-backfill-space/datafile 2020-09-12T14:46:46.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:46.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj312 td/osd-backfill-space/datafile 2020-09-12T14:46:46.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:46.605 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj312 td/osd-backfill-space/datafile 2020-09-12T14:46:46.680 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:46.681 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:46.681 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:46.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj313 td/osd-backfill-space/datafile 2020-09-12T14:46:46.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:46.764 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj313 td/osd-backfill-space/datafile 2020-09-12T14:46:46.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:46.840 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj313 td/osd-backfill-space/datafile 2020-09-12T14:46:46.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:46.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj313 td/osd-backfill-space/datafile 2020-09-12T14:46:46.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:46.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj313 td/osd-backfill-space/datafile 2020-09-12T14:46:47.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:47.075 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj313 td/osd-backfill-space/datafile 2020-09-12T14:46:47.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:47.150 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj313 td/osd-backfill-space/datafile 2020-09-12T14:46:47.233 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:47.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj313 td/osd-backfill-space/datafile 2020-09-12T14:46:47.309 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:47.309 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:47.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:47.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj314 td/osd-backfill-space/datafile 2020-09-12T14:46:47.384 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:47.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj314 td/osd-backfill-space/datafile 2020-09-12T14:46:47.480 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:47.480 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj314 td/osd-backfill-space/datafile 2020-09-12T14:46:47.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:47.569 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj314 td/osd-backfill-space/datafile 2020-09-12T14:46:47.653 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:47.653 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj314 td/osd-backfill-space/datafile 2020-09-12T14:46:47.728 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:47.729 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj314 td/osd-backfill-space/datafile 2020-09-12T14:46:47.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:47.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj314 td/osd-backfill-space/datafile 2020-09-12T14:46:47.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:47.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj314 td/osd-backfill-space/datafile 2020-09-12T14:46:48.030 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:48.030 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:48.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:48.031 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj315 td/osd-backfill-space/datafile 2020-09-12T14:46:48.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:48.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj315 td/osd-backfill-space/datafile 2020-09-12T14:46:48.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:48.181 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj315 td/osd-backfill-space/datafile 2020-09-12T14:46:48.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:48.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj315 td/osd-backfill-space/datafile 2020-09-12T14:46:48.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:48.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj315 td/osd-backfill-space/datafile 2020-09-12T14:46:48.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:48.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj315 td/osd-backfill-space/datafile 2020-09-12T14:46:48.490 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:48.491 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj315 td/osd-backfill-space/datafile 2020-09-12T14:46:48.574 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:48.575 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj315 td/osd-backfill-space/datafile 2020-09-12T14:46:48.649 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:48.650 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:48.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:48.650 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj316 td/osd-backfill-space/datafile 2020-09-12T14:46:48.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:48.800 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj316 td/osd-backfill-space/datafile 2020-09-12T14:46:48.875 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:48.876 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj316 td/osd-backfill-space/datafile 2020-09-12T14:46:48.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:48.951 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj316 td/osd-backfill-space/datafile 2020-09-12T14:46:49.034 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:49.035 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj316 td/osd-backfill-space/datafile 2020-09-12T14:46:49.110 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:49.110 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj316 td/osd-backfill-space/datafile 2020-09-12T14:46:49.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:49.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj316 td/osd-backfill-space/datafile 2020-09-12T14:46:49.269 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:49.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj316 td/osd-backfill-space/datafile 2020-09-12T14:46:49.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:49.345 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:49.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:49.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj317 td/osd-backfill-space/datafile 2020-09-12T14:46:49.420 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:49.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj317 td/osd-backfill-space/datafile 2020-09-12T14:46:49.520 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:49.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj317 td/osd-backfill-space/datafile 2020-09-12T14:46:49.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:49.596 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj317 td/osd-backfill-space/datafile 2020-09-12T14:46:49.671 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:49.672 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj317 td/osd-backfill-space/datafile 2020-09-12T14:46:49.755 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:49.756 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj317 td/osd-backfill-space/datafile 2020-09-12T14:46:49.848 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:49.848 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj317 td/osd-backfill-space/datafile 2020-09-12T14:46:49.923 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:49.924 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj317 td/osd-backfill-space/datafile 2020-09-12T14:46:49.998 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:49.999 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:49.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:49.999 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj318 td/osd-backfill-space/datafile 2020-09-12T14:46:50.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:50.074 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj318 td/osd-backfill-space/datafile 2020-09-12T14:46:50.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:50.149 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj318 td/osd-backfill-space/datafile 2020-09-12T14:46:50.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:50.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj318 td/osd-backfill-space/datafile 2020-09-12T14:46:50.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:50.300 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj318 td/osd-backfill-space/datafile 2020-09-12T14:46:50.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:50.375 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj318 td/osd-backfill-space/datafile 2020-09-12T14:46:50.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:50.451 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj318 td/osd-backfill-space/datafile 2020-09-12T14:46:50.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:50.534 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj318 td/osd-backfill-space/datafile 2020-09-12T14:46:50.610 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:50.610 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:50.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:50.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj319 td/osd-backfill-space/datafile 2020-09-12T14:46:50.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:50.685 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj319 td/osd-backfill-space/datafile 2020-09-12T14:46:50.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:50.763 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj319 td/osd-backfill-space/datafile 2020-09-12T14:46:50.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:50.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj319 td/osd-backfill-space/datafile 2020-09-12T14:46:50.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:50.914 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj319 td/osd-backfill-space/datafile 2020-09-12T14:46:51.008 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:51.009 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj319 td/osd-backfill-space/datafile 2020-09-12T14:46:51.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:51.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj319 td/osd-backfill-space/datafile 2020-09-12T14:46:51.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:51.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj319 td/osd-backfill-space/datafile 2020-09-12T14:46:51.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:51.224 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:51.224 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:51.225 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj320 td/osd-backfill-space/datafile 2020-09-12T14:46:51.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:51.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj320 td/osd-backfill-space/datafile 2020-09-12T14:46:51.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:51.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj320 td/osd-backfill-space/datafile 2020-09-12T14:46:51.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:51.441 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj320 td/osd-backfill-space/datafile 2020-09-12T14:46:51.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:51.567 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj320 td/osd-backfill-space/datafile 2020-09-12T14:46:51.642 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:51.643 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj320 td/osd-backfill-space/datafile 2020-09-12T14:46:51.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:51.718 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj320 td/osd-backfill-space/datafile 2020-09-12T14:46:51.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:51.827 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj320 td/osd-backfill-space/datafile 2020-09-12T14:46:51.902 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:51.902 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:51.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:51.903 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj321 td/osd-backfill-space/datafile 2020-09-12T14:46:51.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:51.978 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj321 td/osd-backfill-space/datafile 2020-09-12T14:46:52.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:52.062 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj321 td/osd-backfill-space/datafile 2020-09-12T14:46:52.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:52.137 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj321 td/osd-backfill-space/datafile 2020-09-12T14:46:52.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:52.212 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj321 td/osd-backfill-space/datafile 2020-09-12T14:46:52.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:52.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj321 td/osd-backfill-space/datafile 2020-09-12T14:46:52.371 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:52.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj321 td/osd-backfill-space/datafile 2020-09-12T14:46:52.438 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:52.439 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj321 td/osd-backfill-space/datafile 2020-09-12T14:46:52.514 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:52.514 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:52.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:52.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj322 td/osd-backfill-space/datafile 2020-09-12T14:46:52.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:52.589 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj322 td/osd-backfill-space/datafile 2020-09-12T14:46:52.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:52.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj322 td/osd-backfill-space/datafile 2020-09-12T14:46:52.731 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:52.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj322 td/osd-backfill-space/datafile 2020-09-12T14:46:52.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:52.816 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj322 td/osd-backfill-space/datafile 2020-09-12T14:46:52.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:52.899 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj322 td/osd-backfill-space/datafile 2020-09-12T14:46:52.991 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:52.992 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj322 td/osd-backfill-space/datafile 2020-09-12T14:46:53.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:53.068 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj322 td/osd-backfill-space/datafile 2020-09-12T14:46:53.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:53.143 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:53.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:53.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj323 td/osd-backfill-space/datafile 2020-09-12T14:46:53.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:53.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj323 td/osd-backfill-space/datafile 2020-09-12T14:46:53.301 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:53.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj323 td/osd-backfill-space/datafile 2020-09-12T14:46:53.376 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:53.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj323 td/osd-backfill-space/datafile 2020-09-12T14:46:53.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:53.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj323 td/osd-backfill-space/datafile 2020-09-12T14:46:53.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:53.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj323 td/osd-backfill-space/datafile 2020-09-12T14:46:53.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:53.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj323 td/osd-backfill-space/datafile 2020-09-12T14:46:53.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:53.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj323 td/osd-backfill-space/datafile 2020-09-12T14:46:53.842 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:53.842 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:53.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:53.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj324 td/osd-backfill-space/datafile 2020-09-12T14:46:53.921 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:53.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj324 td/osd-backfill-space/datafile 2020-09-12T14:46:53.997 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:53.997 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj324 td/osd-backfill-space/datafile 2020-09-12T14:46:54.072 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:54.073 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj324 td/osd-backfill-space/datafile 2020-09-12T14:46:54.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:54.156 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj324 td/osd-backfill-space/datafile 2020-09-12T14:46:54.231 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:54.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj324 td/osd-backfill-space/datafile 2020-09-12T14:46:54.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:54.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj324 td/osd-backfill-space/datafile 2020-09-12T14:46:54.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:54.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj324 td/osd-backfill-space/datafile 2020-09-12T14:46:54.474 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:54.475 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:54.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:54.475 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj325 td/osd-backfill-space/datafile 2020-09-12T14:46:54.558 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:54.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj325 td/osd-backfill-space/datafile 2020-09-12T14:46:54.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:54.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj325 td/osd-backfill-space/datafile 2020-09-12T14:46:54.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:54.709 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj325 td/osd-backfill-space/datafile 2020-09-12T14:46:54.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:54.784 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj325 td/osd-backfill-space/datafile 2020-09-12T14:46:54.859 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:54.860 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj325 td/osd-backfill-space/datafile 2020-09-12T14:46:54.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:54.935 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj325 td/osd-backfill-space/datafile 2020-09-12T14:46:55.010 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:55.011 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj325 td/osd-backfill-space/datafile 2020-09-12T14:46:55.085 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:55.086 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:55.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:55.087 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj326 td/osd-backfill-space/datafile 2020-09-12T14:46:55.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:55.161 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj326 td/osd-backfill-space/datafile 2020-09-12T14:46:55.236 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:55.237 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj326 td/osd-backfill-space/datafile 2020-09-12T14:46:55.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:55.312 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj326 td/osd-backfill-space/datafile 2020-09-12T14:46:55.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:55.387 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj326 td/osd-backfill-space/datafile 2020-09-12T14:46:55.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:55.472 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj326 td/osd-backfill-space/datafile 2020-09-12T14:46:55.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:55.572 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj326 td/osd-backfill-space/datafile 2020-09-12T14:46:55.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:55.647 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj326 td/osd-backfill-space/datafile 2020-09-12T14:46:55.722 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:55.723 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:55.723 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:55.724 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj327 td/osd-backfill-space/datafile 2020-09-12T14:46:55.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:55.890 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj327 td/osd-backfill-space/datafile 2020-09-12T14:46:55.965 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:55.966 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj327 td/osd-backfill-space/datafile 2020-09-12T14:46:56.032 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:56.033 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj327 td/osd-backfill-space/datafile 2020-09-12T14:46:56.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:56.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj327 td/osd-backfill-space/datafile 2020-09-12T14:46:56.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:56.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj327 td/osd-backfill-space/datafile 2020-09-12T14:46:56.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:56.267 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj327 td/osd-backfill-space/datafile 2020-09-12T14:46:56.342 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:56.343 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj327 td/osd-backfill-space/datafile 2020-09-12T14:46:56.418 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:56.418 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:56.418 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:56.419 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj328 td/osd-backfill-space/datafile 2020-09-12T14:46:56.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:56.493 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj328 td/osd-backfill-space/datafile 2020-09-12T14:46:56.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:56.577 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj328 td/osd-backfill-space/datafile 2020-09-12T14:46:56.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:56.652 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj328 td/osd-backfill-space/datafile 2020-09-12T14:46:56.728 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:56.728 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj328 td/osd-backfill-space/datafile 2020-09-12T14:46:56.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:56.803 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj328 td/osd-backfill-space/datafile 2020-09-12T14:46:56.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:56.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj328 td/osd-backfill-space/datafile 2020-09-12T14:46:56.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:56.946 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj328 td/osd-backfill-space/datafile 2020-09-12T14:46:57.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:57.021 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:57.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:57.022 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj329 td/osd-backfill-space/datafile 2020-09-12T14:46:57.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:57.105 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj329 td/osd-backfill-space/datafile 2020-09-12T14:46:57.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:57.180 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj329 td/osd-backfill-space/datafile 2020-09-12T14:46:57.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:57.256 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj329 td/osd-backfill-space/datafile 2020-09-12T14:46:57.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:57.331 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj329 td/osd-backfill-space/datafile 2020-09-12T14:46:57.406 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:57.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj329 td/osd-backfill-space/datafile 2020-09-12T14:46:57.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:57.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj329 td/osd-backfill-space/datafile 2020-09-12T14:46:57.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:57.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj329 td/osd-backfill-space/datafile 2020-09-12T14:46:57.683 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:57.684 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:57.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:57.684 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj330 td/osd-backfill-space/datafile 2020-09-12T14:46:57.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:57.759 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj330 td/osd-backfill-space/datafile 2020-09-12T14:46:57.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:57.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj330 td/osd-backfill-space/datafile 2020-09-12T14:46:57.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:57.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj330 td/osd-backfill-space/datafile 2020-09-12T14:46:58.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:58.038 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj330 td/osd-backfill-space/datafile 2020-09-12T14:46:58.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:58.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj330 td/osd-backfill-space/datafile 2020-09-12T14:46:58.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:58.186 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj330 td/osd-backfill-space/datafile 2020-09-12T14:46:58.261 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:58.261 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj330 td/osd-backfill-space/datafile 2020-09-12T14:46:58.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:58.337 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:58.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:58.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj331 td/osd-backfill-space/datafile 2020-09-12T14:46:58.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:58.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj331 td/osd-backfill-space/datafile 2020-09-12T14:46:58.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:58.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj331 td/osd-backfill-space/datafile 2020-09-12T14:46:58.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:58.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj331 td/osd-backfill-space/datafile 2020-09-12T14:46:58.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:58.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj331 td/osd-backfill-space/datafile 2020-09-12T14:46:58.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:58.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj331 td/osd-backfill-space/datafile 2020-09-12T14:46:58.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:58.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj331 td/osd-backfill-space/datafile 2020-09-12T14:46:58.881 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:58.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj331 td/osd-backfill-space/datafile 2020-09-12T14:46:59.057 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:59.058 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:59.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:59.058 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj332 td/osd-backfill-space/datafile 2020-09-12T14:46:59.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:59.132 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj332 td/osd-backfill-space/datafile 2020-09-12T14:46:59.207 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:59.208 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj332 td/osd-backfill-space/datafile 2020-09-12T14:46:59.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:59.283 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj332 td/osd-backfill-space/datafile 2020-09-12T14:46:59.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:59.359 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj332 td/osd-backfill-space/datafile 2020-09-12T14:46:59.442 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:59.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj332 td/osd-backfill-space/datafile 2020-09-12T14:46:59.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:59.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj332 td/osd-backfill-space/datafile 2020-09-12T14:46:59.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:59.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj332 td/osd-backfill-space/datafile 2020-09-12T14:46:59.710 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:46:59.711 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:46:59.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:59.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj333 td/osd-backfill-space/datafile 2020-09-12T14:46:59.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:59.845 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj333 td/osd-backfill-space/datafile 2020-09-12T14:46:59.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:59.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj333 td/osd-backfill-space/datafile 2020-09-12T14:46:59.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:46:59.995 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj333 td/osd-backfill-space/datafile 2020-09-12T14:47:00.070 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:00.071 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj333 td/osd-backfill-space/datafile 2020-09-12T14:47:00.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:00.146 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj333 td/osd-backfill-space/datafile 2020-09-12T14:47:00.221 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:00.222 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj333 td/osd-backfill-space/datafile 2020-09-12T14:47:00.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:00.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj333 td/osd-backfill-space/datafile 2020-09-12T14:47:00.372 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:00.373 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:00.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:00.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj334 td/osd-backfill-space/datafile 2020-09-12T14:47:00.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:00.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj334 td/osd-backfill-space/datafile 2020-09-12T14:47:00.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:00.523 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj334 td/osd-backfill-space/datafile 2020-09-12T14:47:00.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:00.607 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj334 td/osd-backfill-space/datafile 2020-09-12T14:47:00.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:00.682 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj334 td/osd-backfill-space/datafile 2020-09-12T14:47:00.757 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:00.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj334 td/osd-backfill-space/datafile 2020-09-12T14:47:00.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:00.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj334 td/osd-backfill-space/datafile 2020-09-12T14:47:00.908 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:00.908 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj334 td/osd-backfill-space/datafile 2020-09-12T14:47:00.983 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:00.984 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:00.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:00.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj335 td/osd-backfill-space/datafile 2020-09-12T14:47:01.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:01.067 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj335 td/osd-backfill-space/datafile 2020-09-12T14:47:01.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:01.143 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj335 td/osd-backfill-space/datafile 2020-09-12T14:47:01.226 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:01.227 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj335 td/osd-backfill-space/datafile 2020-09-12T14:47:01.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:01.302 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj335 td/osd-backfill-space/datafile 2020-09-12T14:47:01.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:01.377 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj335 td/osd-backfill-space/datafile 2020-09-12T14:47:01.452 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:01.453 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj335 td/osd-backfill-space/datafile 2020-09-12T14:47:01.561 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:01.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj335 td/osd-backfill-space/datafile 2020-09-12T14:47:01.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:01.637 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:01.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:01.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj336 td/osd-backfill-space/datafile 2020-09-12T14:47:01.712 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:01.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj336 td/osd-backfill-space/datafile 2020-09-12T14:47:01.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:01.847 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj336 td/osd-backfill-space/datafile 2020-09-12T14:47:01.930 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:01.931 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj336 td/osd-backfill-space/datafile 2020-09-12T14:47:02.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:02.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj336 td/osd-backfill-space/datafile 2020-09-12T14:47:02.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:02.081 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj336 td/osd-backfill-space/datafile 2020-09-12T14:47:02.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:02.157 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj336 td/osd-backfill-space/datafile 2020-09-12T14:47:02.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:02.232 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj336 td/osd-backfill-space/datafile 2020-09-12T14:47:02.307 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:02.308 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:02.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:02.308 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj337 td/osd-backfill-space/datafile 2020-09-12T14:47:02.382 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:02.383 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj337 td/osd-backfill-space/datafile 2020-09-12T14:47:02.458 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:02.459 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj337 td/osd-backfill-space/datafile 2020-09-12T14:47:02.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:02.543 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj337 td/osd-backfill-space/datafile 2020-09-12T14:47:02.617 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:02.618 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj337 td/osd-backfill-space/datafile 2020-09-12T14:47:02.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:02.693 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj337 td/osd-backfill-space/datafile 2020-09-12T14:47:02.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:02.768 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj337 td/osd-backfill-space/datafile 2020-09-12T14:47:02.843 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:02.844 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj337 td/osd-backfill-space/datafile 2020-09-12T14:47:02.919 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:02.919 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:02.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:02.920 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj338 td/osd-backfill-space/datafile 2020-09-12T14:47:03.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:03.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj338 td/osd-backfill-space/datafile 2020-09-12T14:47:03.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:03.112 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj338 td/osd-backfill-space/datafile 2020-09-12T14:47:03.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:03.187 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj338 td/osd-backfill-space/datafile 2020-09-12T14:47:03.262 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:03.263 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj338 td/osd-backfill-space/datafile 2020-09-12T14:47:03.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:03.338 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj338 td/osd-backfill-space/datafile 2020-09-12T14:47:03.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:03.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj338 td/osd-backfill-space/datafile 2020-09-12T14:47:03.530 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:03.531 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj338 td/osd-backfill-space/datafile 2020-09-12T14:47:03.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:03.615 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:03.616 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:03.616 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj339 td/osd-backfill-space/datafile 2020-09-12T14:47:03.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:03.690 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj339 td/osd-backfill-space/datafile 2020-09-12T14:47:03.790 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:03.791 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj339 td/osd-backfill-space/datafile 2020-09-12T14:47:03.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:03.866 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj339 td/osd-backfill-space/datafile 2020-09-12T14:47:03.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:03.941 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj339 td/osd-backfill-space/datafile 2020-09-12T14:47:04.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:04.017 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj339 td/osd-backfill-space/datafile 2020-09-12T14:47:04.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:04.159 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj339 td/osd-backfill-space/datafile 2020-09-12T14:47:04.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:04.234 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj339 td/osd-backfill-space/datafile 2020-09-12T14:47:04.310 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:04.311 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:04.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:04.311 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj340 td/osd-backfill-space/datafile 2020-09-12T14:47:04.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:04.385 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj340 td/osd-backfill-space/datafile 2020-09-12T14:47:04.460 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:04.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj340 td/osd-backfill-space/datafile 2020-09-12T14:47:04.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:04.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj340 td/osd-backfill-space/datafile 2020-09-12T14:47:04.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:04.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj340 td/osd-backfill-space/datafile 2020-09-12T14:47:04.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:04.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj340 td/osd-backfill-space/datafile 2020-09-12T14:47:04.771 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:04.771 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj340 td/osd-backfill-space/datafile 2020-09-12T14:47:04.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:04.879 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj340 td/osd-backfill-space/datafile 2020-09-12T14:47:04.954 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:04.955 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:04.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:04.955 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj341 td/osd-backfill-space/datafile 2020-09-12T14:47:05.038 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:05.039 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj341 td/osd-backfill-space/datafile 2020-09-12T14:47:05.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:05.114 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj341 td/osd-backfill-space/datafile 2020-09-12T14:47:05.189 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:05.190 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj341 td/osd-backfill-space/datafile 2020-09-12T14:47:05.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:05.273 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj341 td/osd-backfill-space/datafile 2020-09-12T14:47:05.348 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:05.349 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj341 td/osd-backfill-space/datafile 2020-09-12T14:47:05.423 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:05.424 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj341 td/osd-backfill-space/datafile 2020-09-12T14:47:05.532 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:05.533 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj341 td/osd-backfill-space/datafile 2020-09-12T14:47:05.734 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:05.734 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:05.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:05.735 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj342 td/osd-backfill-space/datafile 2020-09-12T14:47:05.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:05.977 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj342 td/osd-backfill-space/datafile 2020-09-12T14:47:06.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:06.111 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj342 td/osd-backfill-space/datafile 2020-09-12T14:47:06.194 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:06.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj342 td/osd-backfill-space/datafile 2020-09-12T14:47:06.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:06.270 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj342 td/osd-backfill-space/datafile 2020-09-12T14:47:06.345 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:06.346 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj342 td/osd-backfill-space/datafile 2020-09-12T14:47:06.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:06.421 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj342 td/osd-backfill-space/datafile 2020-09-12T14:47:06.487 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:06.488 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj342 td/osd-backfill-space/datafile 2020-09-12T14:47:06.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:06.564 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:06.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:06.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj343 td/osd-backfill-space/datafile 2020-09-12T14:47:06.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:06.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj343 td/osd-backfill-space/datafile 2020-09-12T14:47:06.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:06.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj343 td/osd-backfill-space/datafile 2020-09-12T14:47:06.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:06.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj343 td/osd-backfill-space/datafile 2020-09-12T14:47:06.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:06.865 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj343 td/osd-backfill-space/datafile 2020-09-12T14:47:06.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:06.940 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj343 td/osd-backfill-space/datafile 2020-09-12T14:47:07.015 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:07.016 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj343 td/osd-backfill-space/datafile 2020-09-12T14:47:07.091 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:07.091 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj343 td/osd-backfill-space/datafile 2020-09-12T14:47:07.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:07.167 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:07.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:07.167 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj344 td/osd-backfill-space/datafile 2020-09-12T14:47:07.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:07.242 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj344 td/osd-backfill-space/datafile 2020-09-12T14:47:07.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:07.317 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj344 td/osd-backfill-space/datafile 2020-09-12T14:47:07.392 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:07.393 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj344 td/osd-backfill-space/datafile 2020-09-12T14:47:07.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:07.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj344 td/osd-backfill-space/datafile 2020-09-12T14:47:07.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:07.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj344 td/osd-backfill-space/datafile 2020-09-12T14:47:07.635 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:07.636 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj344 td/osd-backfill-space/datafile 2020-09-12T14:47:07.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:07.711 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj344 td/osd-backfill-space/datafile 2020-09-12T14:47:07.837 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:07.837 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:07.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:07.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj345 td/osd-backfill-space/datafile 2020-09-12T14:47:07.912 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:07.913 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj345 td/osd-backfill-space/datafile 2020-09-12T14:47:07.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:07.988 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj345 td/osd-backfill-space/datafile 2020-09-12T14:47:08.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:08.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj345 td/osd-backfill-space/datafile 2020-09-12T14:47:08.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:08.214 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj345 td/osd-backfill-space/datafile 2020-09-12T14:47:08.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:08.289 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj345 td/osd-backfill-space/datafile 2020-09-12T14:47:08.365 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:08.365 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj345 td/osd-backfill-space/datafile 2020-09-12T14:47:08.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:08.440 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj345 td/osd-backfill-space/datafile 2020-09-12T14:47:08.515 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:08.516 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:08.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:08.516 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj346 td/osd-backfill-space/datafile 2020-09-12T14:47:08.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:08.599 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj346 td/osd-backfill-space/datafile 2020-09-12T14:47:08.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:08.674 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj346 td/osd-backfill-space/datafile 2020-09-12T14:47:08.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:08.758 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj346 td/osd-backfill-space/datafile 2020-09-12T14:47:08.833 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:08.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj346 td/osd-backfill-space/datafile 2020-09-12T14:47:08.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:08.901 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj346 td/osd-backfill-space/datafile 2020-09-12T14:47:08.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:08.976 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj346 td/osd-backfill-space/datafile 2020-09-12T14:47:09.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:09.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj346 td/osd-backfill-space/datafile 2020-09-12T14:47:09.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:09.136 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:09.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:09.136 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj347 td/osd-backfill-space/datafile 2020-09-12T14:47:09.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:09.461 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj347 td/osd-backfill-space/datafile 2020-09-12T14:47:09.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:09.646 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj347 td/osd-backfill-space/datafile 2020-09-12T14:47:09.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:09.741 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj347 td/osd-backfill-space/datafile 2020-09-12T14:47:09.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:09.838 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj347 td/osd-backfill-space/datafile 2020-09-12T14:47:09.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:09.922 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj347 td/osd-backfill-space/datafile 2020-09-12T14:47:10.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:10.006 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj347 td/osd-backfill-space/datafile 2020-09-12T14:47:10.089 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:10.090 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj347 td/osd-backfill-space/datafile 2020-09-12T14:47:10.164 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:10.165 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:10.165 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:10.166 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj348 td/osd-backfill-space/datafile 2020-09-12T14:47:10.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:10.248 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj348 td/osd-backfill-space/datafile 2020-09-12T14:47:10.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:10.332 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj348 td/osd-backfill-space/datafile 2020-09-12T14:47:10.407 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:10.408 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj348 td/osd-backfill-space/datafile 2020-09-12T14:47:10.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:10.483 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj348 td/osd-backfill-space/datafile 2020-09-12T14:47:10.558 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:10.559 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj348 td/osd-backfill-space/datafile 2020-09-12T14:47:10.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:10.634 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj348 td/osd-backfill-space/datafile 2020-09-12T14:47:10.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:10.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj348 td/osd-backfill-space/datafile 2020-09-12T14:47:10.792 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:10.793 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:10.793 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:10.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj349 td/osd-backfill-space/datafile 2020-09-12T14:47:10.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:10.868 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj349 td/osd-backfill-space/datafile 2020-09-12T14:47:10.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:10.952 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj349 td/osd-backfill-space/datafile 2020-09-12T14:47:11.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:11.027 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj349 td/osd-backfill-space/datafile 2020-09-12T14:47:11.102 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:11.103 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj349 td/osd-backfill-space/datafile 2020-09-12T14:47:11.178 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:11.178 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj349 td/osd-backfill-space/datafile 2020-09-12T14:47:11.262 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:11.262 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj349 td/osd-backfill-space/datafile 2020-09-12T14:47:11.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:11.337 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj349 td/osd-backfill-space/datafile 2020-09-12T14:47:11.412 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:11.413 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:11.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:11.413 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj350 td/osd-backfill-space/datafile 2020-09-12T14:47:11.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:11.521 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj350 td/osd-backfill-space/datafile 2020-09-12T14:47:11.613 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:11.614 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj350 td/osd-backfill-space/datafile 2020-09-12T14:47:11.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:11.689 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj350 td/osd-backfill-space/datafile 2020-09-12T14:47:11.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:11.798 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj350 td/osd-backfill-space/datafile 2020-09-12T14:47:11.873 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:11.874 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj350 td/osd-backfill-space/datafile 2020-09-12T14:47:11.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:11.949 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj350 td/osd-backfill-space/datafile 2020-09-12T14:47:12.024 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:12.025 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj350 td/osd-backfill-space/datafile 2020-09-12T14:47:12.108 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:12.109 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:12.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:12.109 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj351 td/osd-backfill-space/datafile 2020-09-12T14:47:12.183 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:12.184 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj351 td/osd-backfill-space/datafile 2020-09-12T14:47:12.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:12.259 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj351 td/osd-backfill-space/datafile 2020-09-12T14:47:12.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:12.334 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj351 td/osd-backfill-space/datafile 2020-09-12T14:47:12.409 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:12.410 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj351 td/osd-backfill-space/datafile 2020-09-12T14:47:12.476 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:12.477 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj351 td/osd-backfill-space/datafile 2020-09-12T14:47:12.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:12.560 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj351 td/osd-backfill-space/datafile 2020-09-12T14:47:12.644 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:12.644 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj351 td/osd-backfill-space/datafile 2020-09-12T14:47:12.719 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:12.720 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:12.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:12.720 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj352 td/osd-backfill-space/datafile 2020-09-12T14:47:12.794 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:12.795 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj352 td/osd-backfill-space/datafile 2020-09-12T14:47:12.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:12.870 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj352 td/osd-backfill-space/datafile 2020-09-12T14:47:12.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:12.945 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj352 td/osd-backfill-space/datafile 2020-09-12T14:47:13.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:13.021 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj352 td/osd-backfill-space/datafile 2020-09-12T14:47:13.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:13.130 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj352 td/osd-backfill-space/datafile 2020-09-12T14:47:13.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:13.205 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj352 td/osd-backfill-space/datafile 2020-09-12T14:47:13.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:13.281 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj352 td/osd-backfill-space/datafile 2020-09-12T14:47:13.356 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:13.357 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:13.357 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:13.358 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj353 td/osd-backfill-space/datafile 2020-09-12T14:47:13.431 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:13.432 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj353 td/osd-backfill-space/datafile 2020-09-12T14:47:13.557 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:13.557 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj353 td/osd-backfill-space/datafile 2020-09-12T14:47:13.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:13.641 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj353 td/osd-backfill-space/datafile 2020-09-12T14:47:13.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:13.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj353 td/osd-backfill-space/datafile 2020-09-12T14:47:13.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:13.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj353 td/osd-backfill-space/datafile 2020-09-12T14:47:13.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:13.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj353 td/osd-backfill-space/datafile 2020-09-12T14:47:13.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:13.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj353 td/osd-backfill-space/datafile 2020-09-12T14:47:14.060 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:14.060 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:14.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:14.061 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj354 td/osd-backfill-space/datafile 2020-09-12T14:47:14.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:14.135 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj354 td/osd-backfill-space/datafile 2020-09-12T14:47:14.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:14.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj354 td/osd-backfill-space/datafile 2020-09-12T14:47:14.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:14.278 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj354 td/osd-backfill-space/datafile 2020-09-12T14:47:14.353 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:14.353 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj354 td/osd-backfill-space/datafile 2020-09-12T14:47:14.428 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:14.429 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj354 td/osd-backfill-space/datafile 2020-09-12T14:47:14.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:14.495 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj354 td/osd-backfill-space/datafile 2020-09-12T14:47:14.562 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:14.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj354 td/osd-backfill-space/datafile 2020-09-12T14:47:14.637 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:14.638 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:14.638 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:14.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj355 td/osd-backfill-space/datafile 2020-09-12T14:47:14.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:14.713 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj355 td/osd-backfill-space/datafile 2020-09-12T14:47:14.788 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:14.789 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj355 td/osd-backfill-space/datafile 2020-09-12T14:47:14.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:14.973 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj355 td/osd-backfill-space/datafile 2020-09-12T14:47:15.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:15.097 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj355 td/osd-backfill-space/datafile 2020-09-12T14:47:15.123 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:15.124 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj355 td/osd-backfill-space/datafile 2020-09-12T14:47:15.198 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:15.199 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj355 td/osd-backfill-space/datafile 2020-09-12T14:47:15.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:15.274 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj355 td/osd-backfill-space/datafile 2020-09-12T14:47:15.350 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:15.350 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:15.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:15.351 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj356 td/osd-backfill-space/datafile 2020-09-12T14:47:15.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:15.425 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj356 td/osd-backfill-space/datafile 2020-09-12T14:47:15.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:15.542 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj356 td/osd-backfill-space/datafile 2020-09-12T14:47:15.626 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:15.626 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj356 td/osd-backfill-space/datafile 2020-09-12T14:47:15.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:15.702 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj356 td/osd-backfill-space/datafile 2020-09-12T14:47:15.818 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:15.819 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj356 td/osd-backfill-space/datafile 2020-09-12T14:47:15.885 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:15.886 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj356 td/osd-backfill-space/datafile 2020-09-12T14:47:15.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:15.961 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj356 td/osd-backfill-space/datafile 2020-09-12T14:47:16.036 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:16.037 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:16.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:16.037 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj357 td/osd-backfill-space/datafile 2020-09-12T14:47:16.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:16.120 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj357 td/osd-backfill-space/datafile 2020-09-12T14:47:16.195 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:16.196 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj357 td/osd-backfill-space/datafile 2020-09-12T14:47:16.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:16.271 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj357 td/osd-backfill-space/datafile 2020-09-12T14:47:16.355 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:16.355 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj357 td/osd-backfill-space/datafile 2020-09-12T14:47:16.430 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:16.431 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj357 td/osd-backfill-space/datafile 2020-09-12T14:47:16.505 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:16.506 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj357 td/osd-backfill-space/datafile 2020-09-12T14:47:16.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:16.581 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj357 td/osd-backfill-space/datafile 2020-09-12T14:47:16.656 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:16.657 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:16.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:16.657 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj358 td/osd-backfill-space/datafile 2020-09-12T14:47:16.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:16.732 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj358 td/osd-backfill-space/datafile 2020-09-12T14:47:16.807 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:16.808 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj358 td/osd-backfill-space/datafile 2020-09-12T14:47:16.882 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:16.883 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj358 td/osd-backfill-space/datafile 2020-09-12T14:47:16.966 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:16.967 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj358 td/osd-backfill-space/datafile 2020-09-12T14:47:17.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:17.042 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj358 td/osd-backfill-space/datafile 2020-09-12T14:47:17.117 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:17.118 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj358 td/osd-backfill-space/datafile 2020-09-12T14:47:17.192 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:17.193 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj358 td/osd-backfill-space/datafile 2020-09-12T14:47:17.276 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:17.277 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:17.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:17.277 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj359 td/osd-backfill-space/datafile 2020-09-12T14:47:17.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:17.352 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj359 td/osd-backfill-space/datafile 2020-09-12T14:47:17.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:17.427 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj359 td/osd-backfill-space/datafile 2020-09-12T14:47:17.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:17.536 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj359 td/osd-backfill-space/datafile 2020-09-12T14:47:17.611 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:17.612 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj359 td/osd-backfill-space/datafile 2020-09-12T14:47:17.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:17.687 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj359 td/osd-backfill-space/datafile 2020-09-12T14:47:17.812 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:17.813 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj359 td/osd-backfill-space/datafile 2020-09-12T14:47:17.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:17.888 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj359 td/osd-backfill-space/datafile 2020-09-12T14:47:17.963 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:17.963 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:17.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:17.964 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj360 td/osd-backfill-space/datafile 2020-09-12T14:47:18.048 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:18.049 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj360 td/osd-backfill-space/datafile 2020-09-12T14:47:18.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:18.148 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj360 td/osd-backfill-space/datafile 2020-09-12T14:47:18.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:18.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj360 td/osd-backfill-space/datafile 2020-09-12T14:47:18.298 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:18.299 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj360 td/osd-backfill-space/datafile 2020-09-12T14:47:18.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:18.374 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj360 td/osd-backfill-space/datafile 2020-09-12T14:47:18.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:18.450 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj360 td/osd-backfill-space/datafile 2020-09-12T14:47:18.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:18.538 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj360 td/osd-backfill-space/datafile 2020-09-12T14:47:18.625 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:18.626 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:18.626 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:18.627 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj361 td/osd-backfill-space/datafile 2020-09-12T14:47:18.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:18.701 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj361 td/osd-backfill-space/datafile 2020-09-12T14:47:18.776 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:18.777 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj361 td/osd-backfill-space/datafile 2020-09-12T14:47:18.851 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:18.852 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj361 td/osd-backfill-space/datafile 2020-09-12T14:47:18.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:18.927 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj361 td/osd-backfill-space/datafile 2020-09-12T14:47:19.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:19.002 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj361 td/osd-backfill-space/datafile 2020-09-12T14:47:19.077 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:19.078 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj361 td/osd-backfill-space/datafile 2020-09-12T14:47:19.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:19.153 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj361 td/osd-backfill-space/datafile 2020-09-12T14:47:19.228 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:19.228 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:19.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:19.229 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj362 td/osd-backfill-space/datafile 2020-09-12T14:47:19.295 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:19.296 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj362 td/osd-backfill-space/datafile 2020-09-12T14:47:19.370 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:19.371 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj362 td/osd-backfill-space/datafile 2020-09-12T14:47:19.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:19.446 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj362 td/osd-backfill-space/datafile 2020-09-12T14:47:19.563 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:19.564 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj362 td/osd-backfill-space/datafile 2020-09-12T14:47:19.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:19.639 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj362 td/osd-backfill-space/datafile 2020-09-12T14:47:19.714 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:19.715 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj362 td/osd-backfill-space/datafile 2020-09-12T14:47:19.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:19.849 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj362 td/osd-backfill-space/datafile 2020-09-12T14:47:19.986 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:19.987 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:19.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:19.987 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj363 td/osd-backfill-space/datafile 2020-09-12T14:47:20.066 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:20.066 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj363 td/osd-backfill-space/datafile 2020-09-12T14:47:20.141 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:20.142 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj363 td/osd-backfill-space/datafile 2020-09-12T14:47:20.217 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:20.218 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj363 td/osd-backfill-space/datafile 2020-09-12T14:47:20.292 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:20.293 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj363 td/osd-backfill-space/datafile 2020-09-12T14:47:20.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:20.368 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj363 td/osd-backfill-space/datafile 2020-09-12T14:47:20.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:20.443 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj363 td/osd-backfill-space/datafile 2020-09-12T14:47:20.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:20.518 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj363 td/osd-backfill-space/datafile 2020-09-12T14:47:20.602 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:20.602 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:20.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:20.603 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj364 td/osd-backfill-space/datafile 2020-09-12T14:47:20.678 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:20.678 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj364 td/osd-backfill-space/datafile 2020-09-12T14:47:20.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:20.753 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj364 td/osd-backfill-space/datafile 2020-09-12T14:47:20.828 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:20.829 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj364 td/osd-backfill-space/datafile 2020-09-12T14:47:20.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:20.904 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj364 td/osd-backfill-space/datafile 2020-09-12T14:47:20.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:20.979 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj364 td/osd-backfill-space/datafile 2020-09-12T14:47:21.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:21.065 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj364 td/osd-backfill-space/datafile 2020-09-12T14:47:21.138 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:21.139 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj364 td/osd-backfill-space/datafile 2020-09-12T14:47:21.222 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:21.223 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:21.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:21.223 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj365 td/osd-backfill-space/datafile 2020-09-12T14:47:21.297 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:21.298 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj365 td/osd-backfill-space/datafile 2020-09-12T14:47:21.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:21.373 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj365 td/osd-backfill-space/datafile 2020-09-12T14:47:21.448 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:21.449 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj365 td/osd-backfill-space/datafile 2020-09-12T14:47:21.557 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:21.557 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj365 td/osd-backfill-space/datafile 2020-09-12T14:47:21.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:21.633 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test6 put obj365 td/osd-backfill-space/datafile 2020-09-12T14:47:21.716 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:21.717 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test7 put obj365 td/osd-backfill-space/datafile 2020-09-12T14:47:21.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:21.834 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test8 put obj365 td/osd-backfill-space/datafile 2020-09-12T14:47:21.909 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:215: TEST_backfill_test_multi: for o in $(seq 1 $objects) 2020-09-12T14:47:21.909 INFO:tasks.workunit.client.0.smithi027.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: seq 1 8 2020-09-12T14:47:21.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:21.910 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test1 put obj366 td/osd-backfill-space/datafile 2020-09-12T14:47:21.984 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:21.985 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test2 put obj366 td/osd-backfill-space/datafile 2020-09-12T14:47:22.051 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:22.052 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test3 put obj366 td/osd-backfill-space/datafile 2020-09-12T14:47:22.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:22.127 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test4 put obj366 td/osd-backfill-space/datafile 2020-09-12T14:47:22.202 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:217: TEST_backfill_test_multi: for p in $(seq 1 $pools) 2020-09-12T14:47:22.203 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-space.sh:219: TEST_backfill_test_multi: rados -p test5 put obj366 td/osd-backfill-space/datafile 2020-09-12T14:47:22.228 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2020-09-12T14:47:22.229 DEBUG:teuthology.task.console_log:Killing console logger for smithi027 2020-09-12T14:47:22.230 DEBUG:teuthology.task.console_log:Killing console logger for smithi027 2020-09-12T14:47:22.230 DEBUG:teuthology.exit:Finished running handlers